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Preface 


The  RAND  Corporation  has  worked  with  the  Army  Medical 
Department  (AMEDD)  on  a  project  entitled  “Implementing  Clinical 
Practice  Guidelines  in  the  Army  Medical  System.”  This  project  was 
undertaken  to  assist  the  AMEDD  in  developing  and  testing  methods 
to  effectively  implement  clinical  practice  guidelines  in  Army  treat¬ 
ment  facilities,  with  the  goal  to  achieve  consistent  and  quality  clinical 
practices  across  the  Army  health  system.  Three  demonstrations  were 
conducted  to  test  and  refine  methods  before  embarking  on  full 
guideline  implementation  across  the  Army  health  system.  These 
demonstrations  tested  use  of  guidelines  for  primary  care  management 
of  low  back  pain,  asthma,  and  diabetes. 

This  report  presents  the  final  findings  from  the  evaluation  that 
RAND  conducted  as  part  of  the  demonstration  for  the  asthma  prac¬ 
tice  guideline,  which  was  conducted  in  1999  and  2000.  The  assess¬ 
ment  included  a  process  evaluation  of  the  experiences  of  the 
participating  military  treatment  facilities  (MTEs)  as  well  as  a  quanti¬ 
tative  analysis  of  clinical  practices.  The  quantitative  analysis  was  per¬ 
formed  to  document  the  extent  to  which  intended  actions  were 
actually  implemented  by  the  MTEs,  assess  short-term  effects  on  clini¬ 
cal  practices,  and  develop  and  test  metrics  and  measurement  methods 
that  can  be  adopted  by  the  AMEDD  for  routine  monitoring  of  pro¬ 
gress. 

We  present  the  findings  from  the  process  evaluation  and  the 
quantitative  analysis  to  provide  as  complete  a  picture  as  possible  of 
baseline  variations  in  practices  across  facilities,  changes  in  clinical 
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practices  made  by  the  demonstration  sites,  and  measurable  effects  of 
these  actions.  We  also  present  diagnostic  information  on  the  quality 
and  limitations  of  available  data  for  monitoring  practice  improve¬ 
ments.  Recommendations  for  future  actions  by  the  AMEDD  are  pre¬ 
sented. 

This  report  is  one  of  three  final  reports  being  generated  in  this 
project.  It  should  be  of  interest  to  anyone  concerned  with  military 
medical  systems  and  policies.  Similar  reports  were  prepared  from  the 
demonstrations  for  the  low  back  pain  and  diabetes  practice  guide¬ 
lines. 

This  research  was  sponsored  by  the  U.S.  Army  Surgeon  General. 
It  was  conducted  jointly  by  RAND  Arroyo  Center,  a  federally  funded 
research  and  development  center  sponsored  by  the  U.S.  Army,  and  by 
the  RAND  Center  for  Military  Health  Policy  Research. 

For  more  information  on  the  RAND  Arroyo  Center,  contact  the 
Director  of  Operations,  (310)  393-0411,  extension  6500,  or  visit  the 
Arroyo  Center’s  Web  site  at  http://www.rand.org/organization/ard/. 
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Summary 


The  Army  Medical  Department  (AMEDD)  has  made  a  commitment 
to  establishing  a  structure  and  process  to  support  its  military  treat¬ 
ment  facilities  (MTFs)  in  implementing  evidence-hased  practice 
guidelines  with  the  goal  of  achieving  best  practices  that  reduce  varia¬ 
tion  and  enhance  quality  of  medical  care.  The  Quality  Management 
Directorate  of  the  Army  Medical  Command  (MEDCOM)  con¬ 
tracted  with  RAND  to  work  as  a  partner  in  the  development  and 
testing  of  guideline  implementation  methods  for  ultimate  application 
to  an  Army-wide  guideline  program. 

Three  practice  guideline  demonstrations  were  fielded  over  a 
two-year  period,  in  each  of  which  participating  Army  MTFs  imple¬ 
mented  a  different  clinical  practice  guideline.  All  the  demonstrations 
worked  with  practice  guidelines  that  were  established  collaboratively 
by  the  Department  of  Veterans  Affairs  (VA)  and  Department  of 
Defense  (DoD). 

This  report  presents  results  from  our  evaluation  of  the  second  of 
the  three  demonstrations,  in  which  four  participating  MTFs  imple¬ 
mented  the  asthma  practice  guideline  in  AMEDD’s  Southeast  Region 
demonstration.'  The  evaluation  included  both  a  process  evaluation  to 
document  the  implementation  activities  of  participating  MTFs,  and 
an  analysis  of  effects  to  estimate  the  extent  to  which  the  sites’  imple¬ 
mentation  activities  affected  specific  measures  of  service  delivery  for 


'  The  first  demonstration  was  for  a  low  back  pain  practice  guideline,  which  was  imple- 
merited  at  four  MTFs  in  the  Great  Plains  Region.  The  third  was  for  a  diabetes  guideline, 
which  was  implemented  by  two  MTFs  in  the  Western  Region. 
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asthma,  with  comparisons  to  a  group  of  similar  MTFs  that  did  not 
implement  the  guideline.  The  evaluation  also  looked  at  the  effects  of 
the  implementation  on  MTF  costs. 


Overview  of  the  Military  Health  System 

The  Army  operates  a  health  system  with  more  than  40  MTFs  across 
the  country  and  overseas  that  provide  health  care  to  military  person¬ 
nel,  their  family  members,  and  retirees.  This  system  has  a  regional 
structure  led  hy  the  Army  Surgeon  General  and  MEDCOM.  The 
MTFs  range  from  small  community  hospitals  to  large  regional  medi¬ 
cal  centers  offering  tertiary  services,  which  provide  both  ambulatory 
care  and  inpatient  services. 

Separate  from  the  military  health  care  system  is  its  health  insur¬ 
ance  program,  called  TRICARE,  that  covers  health  benefits  for  eligi¬ 
ble  military  personnel,  family  members,  and  retirees.  To  augment  the 
MTE  services,  TRICARE  contracts  with  local  community  providers 
in  the  civilian  sector  to  provide  covered  services.  This  insurance  pro¬ 
gram  has  a  managed-care  option  called  TRICARE  Prime.  All  active- 
duty  personnel  are  automatically  enrolled  in  TRICARE  Prime  and 
are  assigned  to  an  MTE-based  primary  care  manager  (PCM),  which 
serves  as  a  gatekeeper  for  all  care.  Military  family  members  and  some 
retirees  also  have  the  option  of  enrolling  in  TRICARE  Prime,  in 
which  case  they  can  choose  either  an  MTE-based  physician  or  a 
community  provider  for  their  PCM.  Those  who  are  eligible  for 
TRICARE  but  choose  not  to  enroll  in  TRICARE  Prime  are  auto¬ 
matically  enrolled  in  another  TRICARE  option  through  which  they 
can  decide  where  to  receive  care  on  a  case-by-case  basis. 


The  Asthma  Practice  Guideline 

The  principal  emphasis  of  the  DoD/VA  practice  guideline  for  pri¬ 
mary  care  management  of  asthma  is  on  effective  management  of 
asthma,  including  medication  management,  with  the  goal  of  pre- 
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venting  exacerbations  that  require  treatment  interventions.  The 
guideline  has  four  key  elements:  initial  asthma  diagnosis;  asthma 
management  procedures  to  classify  asthma  severity,  treat  based  on 
severity,  provide  preventive  maintenance,  and  educate  patients  on 
self-care;  emergency  management  of  asthma  exacerbations;  and  tele¬ 
phone  triage  to  assess  severity  of  exacerbation  and  review  the  action 
plan  with  the  patient. 


Implementation  of  the  Guideline 

Four  MTFs  in  the  Southeast  Regional  Medical  Command  served  as 
demonstration  sites  for  implementation  of  the  asthma  guideline: 
Eisenhower  Army  Medical  Center  (AMC)  at  Fort  Gordon,  Georgia; 
Blanchfield  Army  Community  Hospital  (ACH)  at  Fort  Campbell, 
Kentucky;  Martin  ACH  at  Fort  Benning,  Georgia;  and  Moncrief 
ACH  at  Fort  Jackson,  South  Carolina.  These  four  MTFs  represented 
diverse  patient  populations,  facility  sizes,  and  service  mixes.  In  pre¬ 
paring  for  implementation,  MTF  commanders  designated  a  “guide¬ 
line  champion”  at  each  facility  to  lead  the  implementation  process,  a 
facilitator  to  coordinate  the  MTF’s  implementation  activities,  and  an 
implementation  team  with  representatives  from  the  various  clinical 
groups  involved  in  asthma  care. 

A  systems  approach  was  applied  in  the  AMEDD  practice  guide¬ 
line  implementation  demonstrations.  This  approach  sought  to  ensure 
successful  practice  change  in  MTEs  by  addressing  two  main  dimen¬ 
sions:  building  local  ownership  or  “buy-in”  from  the  staff  responsible 
for  implementing  the  new  practices  and  ensuring  that  clinical  and 
administrative  systems  are  in  place  to  facilitate  staff  adherence  to  the 
guideline. 

The  asthma  guideline  was  introduced  in  September  1999.  To 
prepare  for  implementation,  MEDCOM  held  a  kickoff  conference  to 
introduce  the  implementation  teams  from  participating  MTEs  to  the 
practice  guideline  and  to  provide  monitoring  metrics  and  a  toolkit  of 
materials  to  support  the  MTEs’  implementation  activities.  At  the 
conference,  MTE  teams  developed  action  plans  for  implementing  the 
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guideline.  After  the  conference,  each  MTF  team  began  to  implement 
activities  it  defined  in  its  plan.  Although  the  MTFs  varied  in  how 
quickly  they  started  implementation,  all  of  them  were  pursuing  their 
planned  actions  hy  January  2000. 


The  RAND  Evaluation 

The  evaluation  of  the  asthma  practice  guideline  demonstration  con¬ 
sisted  of  both  a  process  evaluation  and  an  analysis  of  the  effects  of  the 
guideline  on  service  utilization. 

Process  Evaluation.  We  took  a  formative  approach  to  the  pro¬ 
cess  evaluation  in  which  we  learned  from  the  MTFs’  experiences, 
provided  feedback  to  the  MTFs  and  MEDCOM,  and  facilitated 
shared  learning  among  the  MTFs.  To  gather  evaluation  information, 
we  used  a  “climate  survey”  conducted  during  the  kickoff  conference; 
interviews,  focus  groups,  and  surveys,  which  were  conducted  during 
two  evaluation  site  visits^;  and  monthly  progress  reports  prepared  by 
participating  MTFs. 

Effects  Analysis.  The  analysis  of  the  effects  of  the  guideline  on 
service  utilization  used  a  time-series,  comparison-group  design  to 
estimate  effects  of  the  demonstration  on  six  indicators  of  care  that 
could  be  measured  using  available  administrative  data.  These  mea¬ 
sures  and  associated  hypotheses  are  shown  in  Table  S.l. 

We  compared  measures  for  baseline  performance  (one  year 
before  introduction  of  the  asthma  guideline,  January  through  Decem¬ 
ber  1999)  and  performance  at  one  year  following  introduction  (Janu¬ 
ary  through  December  2000)  for  the  four  demonstration  sites  and  six 
control  sites.  We  estimated  MTF  costs  of  care  for  asthma  patients  and 
assessed  how  costs  changed  with  guideline  implementation. 

Each  MTE  provides  asthma  care  not  only  to  patients  enrolled 
with  a  PCM  at  its  facility  but  also  to  patients  enrolled  in  TRICARE 


^  The  first  site  visits  took  place  in  February  and  March  2000.  The  second  site  visits  took 
place  in  September  2000. 
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Table  S.1 

Asthma  Indicators  and  Associated  Hypotheses 


Indicator 

Hypothesis 

Long-term  controllers 

Increase  in  percentage  of  asthma  patients  using  long¬ 
term  controllers  (inhaled  corticosteroid,  leukotriene 
inhibitor,  Betaj  agonist/CS,  or  oral  corticosteroid) 

Complementary  mainte- 

Increase  in  percentage  of  asthma  patients  using 

nance  medications 

complementary  maintenance  medications  (Beta2 
agonist/LA  or  methyixanthine) 

Short-acting  rescue  medi- 

Increase  in  percentage  of  asthma  patients  using  short- 

cations 

acting  rescue  medications  (Betaj  agonist) 

Outpatient  visits 

Decrease  in  asthma-related  outpatient  visit  rate  per 
1,000  asthma  patients 

Emergency  room  visits 

Decrease  in  asthma-related  emergency  room  visit  rate 
per  1,000  asthma  patients 

Hospitalization 

Decrease  in  asthma-related  hospitalization  rate  per 
1,000  asthma  patients 

Prime  with  a  PCM  located  elsewhere  and  others  who  have  chosen  the 
more  open  TRICARE  coverage  option.  Recognizing  this  variety  of 
patients,  three  patient  groups  were  considered  in  the  evaluation  of 
guideline  effects:  the  entire  population  of  TRICARE-eligihle  asthma 
patients  treated  hy  Army  MTFs  or  who  resided  in  Army  MTF 
catchment  areas  and  were  served  hy  network  providers,  all  asthma 
patients  who  used  one  of  the  demonstration  or  control  MTFs  for 
inpatient  or  outpatient  services  at  least  once  during  a  year  {MTF 
users),  and  asthma  patients  who  are  enrolled  in  TRICARE  Prime  and 
have  a  PCM  at  one  of  the  demonstration  or  control  MTFs  {MTF 
enrollees),  who  are  a  subset  of  the  MTF  users. 

The  patient  group  used  as  the  sample  for  assessing  effects  of 
asthma  guideline  implementation  was  the  MTF  enrollees  at  the  dem¬ 
onstration  and  control  MTFs.  The  distinction  between  the  MTF 
enrollees  and  other  patients  served  is  important  for  this  study.  For 
patients  with  such  chronic  diseases  as  asthma,  MTF-based  PCMs 
have  the  span  of  control  to  manage  care  for  the  patients  who  are 
enrolled  with  them.  However,  MTFs  have  much  less  ability  to  man¬ 
age  care  for  patients  they  only  see  intermittently. 
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Baseline  Performance  on  Key  Performance  Measures 

We  first  characterized  the  total  population  of  asthma  patients  (those 
who  used  an  Army  MTF  or  resided  in  an  Army  MTF  catchment  area 
in  the  continental  United  States)  during  the  two-year  study  period.^ 
An  estimated  121,500  asthma  patients  were  served  during  the  first  of 
our  two  study  years  and  an  estimated  121,000  patients  were  served 
during  the  second  study  year.  This  population  consists  primarily  of 
Army  family  members,  individuals  affiliated  with  other  military  ser¬ 
vices,  and  family  members  of  retirees.  Patients  are  fairly  evenly  dis¬ 
tributed  across  age  groups.  The  asthma  populations  served  by  the 
individual  MTFs  vary  widely  in  size,  reflecting  differences  in  the  sizes 
and  characteristics  of  the  beneficiary  populations  residing  in  their 
catchment  areas. 

In  examining  data  for  demonstration  and  control  MTFs,  we  dis¬ 
tinguished  between  MTF  enrollees  and  MTF  users.  Although  the 
majority  of  asthma- related  outpatient  or  emergency  room  visits  were 
for  MTF  enrollees  (patients  who  were  enrolled  in  TRICARE  at  the 
MTF  that  provided  their  care),  a  substantial  portion  of  patients  seen 
were  other  MTF  users  (enrolled  at  other  MTFs  or  civilian  network 
sites).  By  contrast,  the  MTFs’  own  enrollees  accounted  for  virtually 
all  asthma-related  inpatient  care  provided  by  these  MTFs. 

The  baseline  comparisons  of  outcome  measures  for  the  study 
sites  show  that  many  of  the  indicators  varied  only  moderately  across 
the  MTFs  at  baseline.  For  the  three  medication  indicators,  in  par¬ 
ticular,  MTFs  had  similar  percentages  of  patients  using  each  type  of 
medication.  For  some  service-use  indicators,  such  as  emergency  room 
care  and  hospitalization,  one  or  two  MTFs  had  either  much  higher  or 
much  lower  levels  than  the  other  MTFs  in  the  sample.  However,  the 
importance  of  these  differences  depends  on  how  the  actual  perfor¬ 
mance  at  each  site  varies  from  recommended  guidelines,  where  appli¬ 
cable. 


^  This  population  was  defined  using  International  Classification  of  Diseases,  Ninth  Revision 
(ICD-9),  diagnosis  codes  in  administrative  data. 
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Lessons  from  the  Process  Evaluation 

Performance  on  the  Critical  Success  Factors 

Research  on  practice  guideline  implementation  has  documented  that 
a  commitment  to  the  implementation  process,  including  use  of  mul¬ 
tiple  interventions,  is  required  to  achieve  desired  changes  to  clinical 
practices.  Drawing  on  this  literature  as  well  as  the  experiences 
observed  in  the  earlier  AMEDD  low  hack  pain  guideline  demonstra¬ 
tion,  we  identified  six  critical  factors  that  influence  how  successful  an 
MTF  will  he  in  integrating  new  practices  into  its  clinical  and  admin¬ 
istrative  processes.  We  assess  here  the  extent  to  which  MTFs  in  this 
demonstration  realized  these  success  factors,  which  in  turn  affected 
their  progress  in  implementing  practice  improvements. 

•  Command  leadership  commitment  at  the  MTF,  regional, 
and  system  levels.  Management  leaders  at  all  three  levels  of 
AMEDD  influence  how  front-line  personnel  perceive  what  pri¬ 
ority  the  system  places  on  the  use  of  practice  guidelines. 

This  demonstration  had  somewhat  more  positive  support  from 
the  leadership  of  the  participating  MTFs  than  had  been  provided  in 
the  low  back  pain  demonstration,  but  attitudes  by  regional  and 
system-level  leadership  still  were  mixed.  In  the  MTFs,  the  command 
team  supported  the  implementation  teams  as  they  instituted  the 
guideline,  but  this  support  generally  was  passive  and  MTF  com¬ 
manders  did  not  exert  full  ownership  locally. 

•  Monitoring  of  progress.  Both  the  local  MTFs  and  MEDCOM 
have  roles  in  monitoring  the  quality  of  health  care  practices 
according  to  evidence-based  standards  defined  in  practice  guide¬ 
lines  and  roles  in  providing  feedback  needed  for  effective  per¬ 
formance  improvement. 

The  monitoring  activities  in  the  demonstration  had  a  mixed 
track  record.  The  focus  of  the  demonstration  MTFs  was  on  using 
medical  chart  data  to  document  the  extent  to  which  the  new  clinical 
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practices  they  had  introduced  were  in  fact  being  used — e.g.,  to 
document  asthma  severity  in  the  chart.  Such  a  focus  helps  to  ensure 
that  these  practices  are  becoming  an  integral  part  of  clinic  processes  as 
intended.  Other  than  the  analysis  performed  in  this  evaluation, 
MEDCOM  did  not  monitor  asthma  metrics  during  the  demonstra¬ 
tion  but  relied  on  data  generated  by  the  MTFs. 

•  Guidance  and  support  to  the  MTFs  by  MEDCOM.  The 

structured  approach  and  toolkits  of  supportive  materials  pro¬ 
vided  are  resources  that  support  the  MTFs  as  they  carry  out 
actions  to  improve  clinical  practices. 

By  the  time  the  asthma  guideline  demonstration  began, 
MEDCOM  had  expanded  its  staffing  and  other  resources,  and  we 
observed  its  staff  providing  regular  policy  guidance  and  technical 
support  to  help  the  MTF  teams  implement  practice  improvements 
for  asthma  care.  During  the  site  visits,  the  implementation  teams  at 
the  demonstration  MTFs  reported  this  committed  MEDCOM  sup¬ 
port  was  helpful  to  them  and  they  were  pleased  to  have  it. 

•  Guideline  champions  who  are  opinion  leaders.  There  is 
extensive  evidence  of  the  importance  of  having  a  designated 
clinical  leader  to  serve  as  champion  for  the  practice  improve¬ 
ments  being  pursued. 

The  participating  MTFs  identified  well-respected  physicians  to 
serve  as  guideline  champions  for  the  asthma  demonstration,  and  these 
physicians  showed  a  commitment  to  leading  the  implementation 
activities  for  their  facilities.  However,  the  champions  could  only  make 
a  time-limited  commitment  to  the  initiative,  after  which  they  tired  of 
the  concentrated  effort  or  had  to  turn  their  attention  to  other  priori¬ 
ties. 


•  Resource  support  for  champions.  To  serve  effectively  as  a 
guideline  champion,  the  designated  champion  needs  to  be  given 
adequate  dedicated  time  and  other  resource  support.  This  sup- 
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port  also  signals  that  the  MTF  command  places  a  priority  on 
guideline  implementation. 

The  MTF  commanders  did  not  provide  tangible  resource  sup¬ 
port  for  the  activities  of  the  guideline  champions,  other  than  for 
attendance  at  the  kickoff  conference.  As  a  result,  the  champions  per¬ 
formed  the  implementation  work  in  addition  to  their  regular  work¬ 
load,  which  contributed  to  their  reluctance  to  sustain  the  champion 
role.  Facilitators  designated  by  the  MTF  commander  provided  some 
staff  support  for  the  champions,  a  role  that  was  part  of  the  facilitators’ 
regular  responsibilities  because  they  worked  in  the  MTF  quality  man¬ 
agement  offices.  The  delayed  implementation  of  the  MTF  action 
plans  stemmed  in  part  from  competing  demands  on  the  champions’ 
time. 


•  Institutionalization  of  new  practices.  For  sustainability,  the 
new  practices  being  introduced  need  to  be  integrated  into  the 
standard  practices  of  the  facility  as  quickly  as  possible. 

At  the  time  of  the  last  process  evaluation  site  visit,  the  partici¬ 
pating  MTFs  had  made  progress  in  introducing  improved  asthma 
management  practices  in  some  of  their  primary  care  clinics,  but  they 
had  not  yet  achieved  sustainable  practices  in  those  clinics.  None  had 
yet  begun  to  extend  the  new  practices  into  other  clinics  serving 
asthma  patients  that  had  not  participated  in  the  demonstration. 

In  summary,  we  observed  reasonably  good  performance  on  some 
of  the  success  factors  in  this  demonstration.  The  most  noticeable 
positive  items  were  the  MTF  efforts  to  monitor  their  progress  in 
implementing  the  intended  practice  changes  and  selection  of  effective 
champions.  MEDCOM  also  was  able  to  provide  responsive  support 
for  the  asthma  demonstration.  It  appears  that  lessons  learned  from 
the  earlier  low  back  pain  demonstration  contributed  to  these  man¬ 
agement  results  (see  Farley,  Vernez,  et  ah,  2003).  Although  the  par¬ 
ticipating  MTFs  identified  effective  champions,  the  champions  were 
not  given  dedicated  time  to  help  them  perform  their  additional  roles. 
Competing  demands  on  champions’  time  weakened  the  teams’ 
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actions  to  introduce  and  sustain  improved  clinical  practices,  as  well  as 
effects  on  clinical  practice  indicators. 

Other  Lessons  from  the  Demonstration 

Other  lessons  learned  from  the  implementation  include  the  follow¬ 
ing. 

Strategies.  The  MTF  were  given  the  flexibility  to  design  strate¬ 
gies  that  best  met  their  needs.  They  used  it  to  emphasize  different 
components  of  the  guideline  and  to  undertake  a  variety  of  actions  for 
change.  Some  risk  is  involved  in  this  approach,  however,  that  a  team 
might  pursue  only  expedient  actions  that  are  not  resisted  by  clinical 
or  administrative  staff,  which  would  slow  progress  toward  the 
achievement  of  consistent  practices  across  the  AMEDD  system. 

Monitoring.  Although  the  MTF  teams  took  initiative  to  monitor 
asthma  measures  during  the  demonstration,  several  issues  arose  that 
require  further  attention.  The  data  collected  by  the  MTF  teams  were 
neither  communicated  to  clinic  staff  to  give  them  empirical  knowl¬ 
edge  of  their  performance  on  key  aspects  of  care  nor  used  to  create 
accountability  for  performance.  Measurement  issues  also  were  identi¬ 
fied,  including  difficulty  in  retrieving  administrative  data  the  MTFs 
needed  for  monitoring,  inconsistencies  in  chart  abstraction  processes, 
and  inaccurate  coding  of  asthma  visits. 

Standard  Forms.  The  standard  asthma  encounter  documenta¬ 
tion  form  developed  by  MEDCOM  received  mixed  reactions  by  pro¬ 
viders  because  the  form  did  not  fully  meet  their  needs.  Because 
MEDCOM  made  the  use  of  the  form  voluntary,  many  MTFs  and 
primary  care  providers  chose  not  to  use  the  form,  preferring  to 
develop  and  use  their  own  forms.  Inconsistent  use  of  the  form  makes 
it  difficult  to  monitor  performance  because  the  needed  data  are 
incomplete. 

Provider  Training.  The  MTFs  learned  that  multiple  and  ongoing 
training  sessions  would  be  required  over  time  to  train  all  primary  care 
providers  effectively  on  the  asthma  management  processes  specified  in 
the  guideline.  The  first  training  sessions  reached  only  a  fraction  of  the 
MTF  providers,  and  continued  training  also  was  needed  to  refresh 
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their  knowledge  and  to  train  newly  arrived  providers  who  rotated  in 
from  previous  MTF  assignments. 

Patient  Education.  The  provision  of  patient  education  on  self- 
care  was  one  of  the  weaker  components  of  the  implementation  activi¬ 
ties.  Patient  behaviors  affect  the  MTFs’  ability  to  achieve  the 
intended  asthma  care  practices  and  outcomes.  How  a  patient  handles 
the  preventive  aspects  of  asthma  management  will  influence  the  fre¬ 
quency  and  severity  of  asthma  exacerbations.  Inadequacies  in  MTF 
patient  education  activities  were  identified,  including  problems  with 
program  design,  limited  receptivity  of  providers  to  referring  patients 
for  education,  and  limited  patient  attendance  at  the  programs  when 
referred. 


Effects  of  the  Demonstration  on  Service  and  Costs 

Effects  on  Performance  Measures 

The  RAND  analysis  found  no  signiflcant  changes  in  the  six  clinical 
practice  indicators  we  identified  for  evaluating  the  effects  of  the 
asthma  practice  guideline  demonstration.  All  three  indicators  for  use 
of  asthma  medications  declined  from  the  first  to  second  study  year, 
which  was  the  reverse  of  the  hypothesized  direction  of  change.  Out¬ 
patient  visit  rates  for  the  demonstration  MTFs  did  not  change  from 
the  first  to  second  year,  although  we  did  observe  seasonal  variations  in 
rates.  For  emergency  room  visit  rates  and  hospitalization  rates — 
which  represent  potentially  avoidable  health-care  events  that  should 
decline  as  asthma  management  improved — we  found  no  changes  in 
rates  during  the  demonstration. 

There  could  be  several  explanations  for  these  null  findings.  The 
most  obvious  is  that  the  practice  improvements  the  MTFs  imple¬ 
mented  were  not  sufficient  to  achieve  changes  in  the  measures.  How¬ 
ever,  it  also  may  be  too  early  to  detect  some  changes,  such  as  reduced 
hospitalization  rates.  Alternatively,  other  practice  changes  might  have 
occurred  within  the  health-care  encounters  that  were  not  captured 
adequately  in  these  measures  of  encounter  frequency.  For  example, 
opposing  effects  might  be  interacting  in  which  better  classification  of 
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asthma  severity  moved  more  patients  to  mild  intermittent  levels, 
which  would  offset  reclassifications  to  higher  severity  levels  with 
unpredictable  net  effects  on  use  of  asthma  medications. 

Patterns  and  Trends  in  MTF  Costs 

The  analysis  of  MTF  costs  revealed  a  decrease  between  the  first  and 
second  study  years  in  per-patient  costs  for  the  MTF  enrollees  at  the 
demonstration  MTFs,  after  adjusting  for  cost  trends  for  the  control 
MTFs  (which  control  for  temporal  effects  on  use  rates).  Despite  our 
inability  to  observe  changes  in  the  indicators  we  were  tracking,  it  is 
possible  that  early  practice  changes  made  by  the  MTFs  in  introducing 
the  practice  guideline  may  have  decreased  the  costs  of  care  for 
enrolled  patients  served  by  the  MTFs.  If  their  actions  did  contribute, 
a  likely  source  of  effects  would  be  changes  in  outpatient  service  mix 
or  in  the  intensity  of  care  during  hospitalizations.  However,  other 
factors  might  also  be  contributing  to  the  changes  in  service-use  pat¬ 
terns  that  led  to  the  observed  cost  reductions.  Inpatient-use  rates  and 
costs  should  be  tracked  over  time  to  identify  trends  and  longer-term 
effects,  as  new  care  management  methods  become  stronger. 


Data  Issues 

Accurate  assessment  of  MTFs’  performance  in  implementing  treat¬ 
ment  guidelines  requires  the  capability  to  routinely  generate  accurate 
and  reliable  data  on  the  indicators  monitored.  Pertinent  to  this  need, 
we  identified  three  critical  data  issues  that  need  to  be  addressed: 

•  Inconsistent  coding  of  diagnoses  and  procedures.  Effective 
monitoring  of  performance  in  treating  asthma  (or  some  other 
condition)  requires  consistent  coding  of  diagnoses  and  proce¬ 
dures  in  the  outpatient  encounter  records.  MEDCOM  has 
established  standard  codes  for  asthma,  but  at  the  time  of  the 
demonstration  these  codes  had  just  been  introduced  and  were 
not  used  consistently  by  the  demonstration  MTFs. 
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•  Unavailable  data.  At  the  system  level,  the  data  needed  to  cal¬ 
culate  many  indicators  (e.g.,  laboratory  or  radiology  data)  were 
incomplete,  were  obtained  from  separate  data-extraction  pro¬ 
cesses  of  varying  quality,  or  were  not  currently  available. 

•  Absence  of  an  asthma  registry.  The  Army  health  system  lacks  a 
centralized  registry  that  can  provide  complete  information  on  all 
asthma  patients  in  the  system  and  can  be  accessed  by  MTFs 
wherever  they  may  be.  In  the  absence  of  this  data  resource, 
asthma  patients  might  not  be  identified  or  information  on  their 
past  care  and  asthma  status  might  be  lost  as  personnel  and  their 
families  move  to  new  locations. 


Recommendations 

Ultimately,  a  practice  guideline  cannot  be  said  to  be  implemented 
until  lasting  changes  in  practices  are  made.  Yet  all  of  the  MTFs  par¬ 
ticipating  in  this  demonstration  had  difficulty  integrating  the  new 
practices  into  the  normal,  ongoing  MTF  clinic  operation.  This  find¬ 
ing  highlights  the  need  for  focused  attention  by  the  leadership  of 
MEDCOM  and  the  MTFs  to  communicate  clearly  that  achieving 
best  practices  is  a  system  priority.  It  also  highlights  the  need  to  con¬ 
tinue  to  reinforce  MTFs’  implementation  activities  through  technical 
support  and  effective  monitoring  to  provide  feedback  to  the  MTFs 
on  their  progress. 

We  summarize  here  our  recommendations  for  improving  the 
implementation  of  the  asthma  guideline. 

•  MEDCOM  needs  to  establish  consistent  monitoring  stan¬ 
dards  for  performance  metrics.  To  achieve  this  consistency, 
standardized  coding  for  patient  status  or  procedures  will  need  to 
be  implemented  effectively  across  the  Army  MTEs.  MEDCOM 
will  also  need  to  consider  whether  it  wants  to  establish  a  cen¬ 
tralized  system  to  collect  the  data  directly  from  automated  data 
systems  or  to  have  MTEs  collect  and  analyze  data  locally  and 
then  report  to  MEDCOM. 
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•  MEDCOM  should  work  with  the  MTFs  to  establish  per¬ 
formance  objectives  on  the  asthma  metrics.  To  ensure  that 
performance  information  is  used  to  improve  clinical  practices, 
monitoring  of  the  asthma  metrics  should  he  integrated  in  the 
MTFs’  quality  management  or  peer  review  programs  and  the 
MTF  commanders  should  review  processes  and  results  regularly. 

•  MEDCOM  should  develop  software  programs  necessary  to 
allow  the  MTEs  to  retrieve  Composite  Health-Care  System 
(CHCS)  and  Ambulatory  Data  System  (ADS)  data.  MTFs 
currently  have  difficulty  retrieving  ADS  and  CHCS  data  for  use 
in  the  monitoring  process.  To  address  this  difficulty,  the  MTFs 
requested  that  MEDCOM  provide  them  with  the  “ad  hoc” 
software  programs  needed  to  extract  the  data. 

•  As  MEDCOM  monitors  the  asthma  metrics  across  MTEs,  it 
needs  to  identify  where  improvements  in  quality  and  consis¬ 
tency  of  care  are  needed.  The  MTFs  were  given  considerable 
flexihility  to  develop  implementation  strategies.  While  this  flexi¬ 
bility  helps  to  ensure  that  each  team  can  address  the  clinical 
practices  most  in  need  of  improvement  at  its  own  MTF,  it  can 
also  risk  slowing  progress  toward  the  AMEDD  goal  of  achieving 
consistent  practices  across  facilities.  By  continuing  to  monitor 
the  metrics  closely  over  time,  MEDCOM  can  determine 
whether  to  give  greater  direction  to  MTFs  regarding  which 
aspects  of  the  guideline  are  to  be  emphasized  and  implemented 
uniformly. 

•  MEDCOM  needs  to  establish  clear  procedures  and  expecta¬ 
tions  for  the  use  of  forms.  Although  sites  were  told  that  the  use 
of  the  forms  provided  by  MEDCOM  was  voluntary,  partici¬ 
pants  at  some  of  the  MTFs  still  thought  that  use  of  the  forms 
was  mandatory.  Other  sites  chose  not  to  use  the  forms,  but  they 
did  not  apply  alternative  methods  to  ensure  that  asthma  diagno¬ 
sis  and  treatment  were  being  documented  appropriately. 
MEDCOM  needs  to  forge  a  policy  regarding  the  use  of  forms 
that  supports  efficiency  and  value  for  providers  and  patients, 
particularly  for  patients  with  multiple  conditions  for  which 
more  than  one  guideline  may  apply. 
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MEDCOM  needs  to  further  define  the  role  of  patient  educa¬ 
tion  in  treatment  processes  of  chronic  conditions,  while 
MTFs  need  to  ensure  that  they  are  using  the  most  effective 
patient  education  techniques.  The  issue  of  patient  education 
has  increased  in  salience  for  AMEDD  because  many  of  the 
guidelines  it  has  implemented  are  for  chronic  conditions  that 
require  self-care  management  hy  patients  for  effective  overall 
management  of  the  condition.  MEDCOM  needs  to  establish 
clear  standards  for  patient  education  and  ensure  that  MTFs  have 
adequate  resources  and  tested  educational  methods. 

MTFs  need  to  integrate  training  on  clinical  guidelines  into 
their  ongoing  education  for  existing  personnel  as  well  as  into 
the  orientation  sessions  for  both  incoming  primary  care  pro¬ 
viders  and  ancillary  staff,  Implementation  teams  often  found 
that  the  training  session  on  guideline  implementation  turned 
into  a  discussion  of  whether  to  implement  the  guideline  rather 
than  how  to  implement  it.  To  train  all  primary  care  providers  to 
desired  levels  of  knowledge,  multiple  and  ongoing  training  ses¬ 
sions  would  clearly  be  required  over  time,  as  providers  deployed 
or  rotated  in  and  out  of  the  MTFs. 

MTFs  need  to  integrate  new  practices  into  normal  clinic 
operation — i.e.,  the  way  they  “do  business”  for  patient  care. 

A  practice  guideline  cannot  be  said  to  be  implemented  until 
such  lasting  changes  in  practices  are  made.  To  help  MTFs  make 
lasting  practice  improvements,  MEDCOM  needs  to  communi¬ 
cate  clearly  that  achieving  best  practices  is  a  system  priority,  and 
it  should  continue  to  support  and  reinforce  the  MTFs’  efforts  by 
providing  technical  support  and  establishing  an  effective  moni¬ 
toring  system  to  track  and  provide  feedback  to  the  MTFs  on 
their  progress. 
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CHAPTER  ONE 

Introduction 


The  Army  Medical  Department  (AMEDD)  has  made  a  commitment 
to  establishing  a  structure  and  process  to  support  its  military  treat¬ 
ment  facilities  (MTFs)  in  implementing  evidence-hased  practice 
guidelines  to  achieve  best  practices  that  reduce  variation  and  enhance 
quality  of  medical  care.  With  the  goal  of  establishing  such  a  system, 
the  AMEDD  contracted  with  RAND  to  work  as  a  partner  in  the 
development  and  testing  of  guideline  implementation  methods  for 
ultimate  application  to  an  Army-wide  guideline  program. 

The  AMEDD-RAND  project  fielded  three  sequential  demon¬ 
strations  over  a  two-year  period,  in  each  of  which  participating  MTFs 
implemented  a  different  clinical  practice  guideline.  This  approach 
was  taken  to  enable  AMEDD  to  test  and  refine  new  implementation 
methods  on  a  small  scale  and  then  apply  these  methods  to  roll  out 
practice  guidelines  across  the  Army  health  system. 

Each  of  the  three  demonstrations  used  a  practice  guideline  that 
was  established  collaboratively  by  the  Department  of  Veterans  Affairs 
(VA)  and  Department  of  Defense  (DoD).  In  the  first  demonstration, 
a  low  back  pain  practice  guideline  was  implemented  in  four  MTFs  in 
the  Great  Plains  Region  (Farley,  Vernez,  et  al.,  2003).  In  the  second 
demonstration,  which  is  the  subject  of  this  report,  an  asthma  guide¬ 
line  was  implemented  by  two  MTFs  in  the  Southeast  Region.  In  the 
final  demonstration,  a  diabetes  guideline  was  implemented  by  two 
MTFs  in  the  Western  Region  (Farley  et  al.,  2005). 

For  each  demonstration,  RAND  performed  a  process  evaluation 
of  the  implementation  process  and  an  assessment  of  the  effects  of  the 
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implementation  on  service  use  at  participating  MTFs.  The  evaluation 
also  documented  the  measurement  methods  and  related  data  require¬ 
ments  to  provide  a  basis  for  future  systemwide  monitoring  of  progress 
in  achieving  best  practices  for  each  condition  addressed  by  a  guide¬ 
line.  This  report  documents  lessons  learned  from  the  evaluation. 

In  the  remainder  of  this  chapter,  we  provide  an  overview  of  the 
military  health  system  and  we  summarize  the  process  that  DoD  and 
the  VA  used  to  establish  practice  guidelines  and  the  approach  used  by 
the  Army’s  Medical  Command  (MEDCOM)  to  implement  the 
guidelines  in  the  Army  health  system. 


Overview  of  the  Military  Health  System 

The  Army’s  health  system  includes  more  than  40  MTFs  operating 
across  the  country  and  overseas.  The  MTFs  provide  medical  care  to 
active-duty  military  personnel  from  all  services  and  their  family 
members  as  well  as  military  retirees.  This  system  has  a  regional  struc¬ 
ture  led  by  the  Army  Surgeon  General  and  MEDCOM.  The  MTFs 
range  from  small  community  hospitals  to  large  regional  medical  cen¬ 
ters  that  offer  tertiary  services.  Physicians  provide  care  to  their 
patients  at  clinics  within  the  MTF.  MTFs  provide  both  ambulatory 
care  and  inpatient  services,  including  diagnostic  services  for  both 
clinics  and  inpatient  units.  Smaller  facilities  refer  complex  cases  to  the 
military  medical  center  serving  the  region  in  which  they  are  located. 

The  MTFs  play  an  important  role  in  the  military’s  health  insur¬ 
ance  program,  known  as  TRICARE,  and  its  managed  care  option, 
known  as  TRICARE  Prime.  Coverage  under  TRICARE  Prime  is 
provided  through  the  MTFs  as  well  as  through  local  civilian  providers 
under  contract.  All  active-duty  personnel  are  automatically  enrolled 
in  TRICARE  Prime  and  are  assigned  to  an  MTF-based  primary  care 
manager  (PCM),  who  serves  as  a  gatekeeper  for  all  care.  Military 
family  members  and  some  retirees  also  have  the  option  of  enrolling  in 
TRICARE  Prime  and  can  choose  either  an  MTF-based  physician  or  a 
community  provider  for  their  PCM.  Family  members  who  do  not 
choose  to  enroll  in  TRICARE  Prime  are  by  default  considered  to  be 
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enrolled  in  TRICARE  Standard/Extra,  which  allows  heneficiaries  to 
decide  where  to  receive  care  on  a  case-hy-case  basis  (Earley,  Harris,  et 
al.,  2003). 

The  structure  of  TRICARE  Prime  has  implications  for  the 
delivery  of  asthma  care  hy  MTEs,  and  it  influenced  the  design  of  our 
evaluation  of  guideline  effects.  Each  MTE  serves  both  patients  who 
are  enrolled  in  TRICARE  Prime  with  PCMs  at  its  facility  and 
patients  who  are  enrolled  in  TRICARE  Prime  with  a  PCM  elsewhere 
or  who  have  chosen  the  more  open  TRICARE  coverage  option.  Eor 
patients  with  such  chronic  diseases  as  asthma,  MTE-based  PCMs 
have  the  span  of  control  to  manage  care  for  the  patients  who  are 
enrolled  with  them.  However,  MTEs  have  much  less  ability  to  man¬ 
age  care  for  patients  they  see  intermittently.  Therefore,  we  focused 
our  evaluation  of  guideline  effects  on  TRICARE  Prime  patients 
enrolled  with  PCMs  at  the  participating  MTEs  because  we  would 
expect  that  improvements  in  management  of  asthma  care  should  have 
the  greatest  impact  on  this  group.' 


The  DoD/VA  Guideline  Adaptation  Process 

The  DoD  and  VA  initiated  a  collaborative  project  in  early  1998  to 
establish  a  single  standard  of  care  throughout  the  military  and  VA 
health  systems.  This  project  is  led  by  a  working  group  consisting  of 
two  representatives  from  each  of  the  three  military  services  and  the 
VA.  The  goals  of  this  project  are  adaptation  of  existing  clinical  prac¬ 
tice  guidelines  for  selected  conditions,  selection  of  two  to  four  indica¬ 
tors  for  each  guideline  to  benchmark  and  monitor  implementation 
progress,  and  integration  of  DoD/VA  prevention,  pharmaceutical, 
and  informatics  efforts. 

Eor  each  practice  guideline,  the  DoD/VA  Working  Group  des¬ 
ignates  an  expert  panel  of  representatives  from  the  three  military 


'  Other  MTF  patients,  including  TRICARE  Prime  patients  who  use  a  participating  MTF 
but  are  not  enrolled  there  and  the  total  population  of  asthma  patients  in  the  military  health 
system,  are  also  discussed  in  the  study  to  provide  points  of  comparison. 
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services  and  the  VA  who  represent  a  mix  of  clinical  backgrounds  rele¬ 
vant  to  the  health  condition  of  interest.  The  expert  panel  reviews 
existing  national  guidelines  for  that  condition  and  examines  and 
updates  the  scientific  evidence  supporting  the  national  guidelines  to 
establish  a  guideline  for  the  military  and  veteran  health  systems.  Each 
panel  also  recommends  metrics  to  be  used  to  monitor  progress  in 
guideline  implementation. 

Each  DoD/VA  practice  guideline  is  a  statement  of  best  practices 
for  the  management  and  treatment  of  the  health  condition  it 
addresses,  and  it  takes  into  account  the  strength  of  relevant  scientific 
evidence,  which  is  documented  in  the  practice  guideline  report.  The 
guidelines  identify  specific  practices  that  are  either  strongly  recom¬ 
mended  or  not  recommended,  while  supporting  clinical  discretion  on 
the  part  of  providers. 


The  AMEDD-RAND  Guideline  Implementation  Project 

The  three  sequential  demonstrations  for  low  back  pain,  asthma,  and 
diabetes  guidelines  have  allowed  AMEDD,  RAND,  and  the  partici¬ 
pating  MTEs  to  test  and  refine  implementation  methods.  As  shown 
in  Eigure  1.1,  each  demonstration  was  part  of  a  “continuous  quality 
improvement”  cycle  through  which  a  regional  test  preceded  system- 
wide  implementation  of  a  practice  guideline.  As  the  demonstrations 
progressed,  RAND  performed  process  evaluations  to  learn  from  the 
experiences  of  participating  MTEs,  and  the  cumulative  results  of  past 
evaluations  guided  preparation  for  each  subsequent  demonstration. 
While  the  evaluations  were  under  way,  MEDCOM  began  prepara¬ 
tions  to  implement  the  guideline  in  all  MTEs  across  the  Army  health 
system. 

The  DoD/VA  low  back  pain  guideline  was  introduced  in  the 
Great  Plains  Regional  Medical  Command  in  November  1998,  while 
the  asthma  practice  guideline  demonstration  was  introduced  in  the 
Southeast  Regional  Medical  Command  in  August  1999  and  the  dia¬ 
betes  guideline  was  introduced  in  the  Western  Regional  Medical 
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Figure  1.1 

Diagram  of  the  Demonstration  Project 


Oct '98  Jan '99  Apr '99  Jul '99  Oct '99  Jan '00  Apr '00  Jul '00 

NOTE:  Two  MTFs  participated  in  the  diabetes  demonstration,  and  data  for  an 
additional  three  MTFs  that  also  implemented  this  guideline  were  used  in  the 
analysis  of  effects  of  implementing  the  guideline. 
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Command  in  December  1999.  Army-wide  implementation  began  in 
spring  2000  for  the  low  back  pain  guideline,  in  Fall  2000  for  the 
asthma  guideline,  and  in  early  200 1  for  the  diabetes  guideline. 


Overview  of  the  Asthma  Practice  Guideline 

Separate  DoD/VA  practice  asthma  guidelines  were  established  for 
primary  care  management  of  pediatric  and  adult  asthma.  The  princi¬ 
pal  emphasis  of  both  practice  guidelines  is  on  effective  management 
of  asthma,  including  medication  management,  with  the  goal  of  pre¬ 
venting  exacerbations  that  require  treatment  interventions.  Four  key 
elements  of  the  guidelines  are  presented  in  Table  1.1. 

As  the  MTFs  implemented  the  asthma  guidelines,  any  resulting 
changes  in  clinical  practices  were  expected  to  reflect  the  guideline’s 
emphasis  on  effective  identification  of  probable  asthma  patients  and 
management  of  the  asthma  to  reduce  the  frequency  of  asthma  exacer¬ 
bations.  To  the  extent  that  MTFs  strengthened  these  practices 
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Table  1.1 

Key  Elements  of  the  DoD/VA  Asthma  Practice  Guidelines 


Key  Element 

Description 

Initial  Asthma  Diagnosis 

Establish  asthma  diagnosis. 

Consider  asthma  in  the  differential  diagnosis  for 
patients  presenting  with  respiratory  problems, 
using  spirometry  and  trials  of  medications  to  test 
for  asthma  as  a  diagnosis. 

Asthma  Management  Procedures  in  Follow-Up  Visits 

Classify  asthma  severity,  treat 
based  on  severity,  provide 
preventive  maintenance,  and 
educate  patients  on  self-care. 

Classify  severity  using  National  Heart,  Lung,  and 
Blood  Institute  (NHLBI)  standards  and  objective 
measures  (spirometry  or  peak  flow)  and  patient 
report  of  symptoms.  Treat  based  on  severity  with 
step-care  approach;  provide  quick  relievers  and 
long-term  controllers.  Educate  patients  to  self¬ 
monitor  with  peak  flow  meter,  use  medications, 
and  recognize  symptoms  of  worsening  asthma. 

Use  written  action  plans,  preventive  mainte¬ 
nance,  and  trigger  avoidance.  Follow  up  regu¬ 
larly. 

Emergency  Management  of  Asthma  Exacerbations 

Perform  initial  objective 
assessment,  treat  promptly, 
and  discharge  patient  with 
education. 

When  patient  reports  exacerbation,  perform 
objective  assessment  using  pulse  oximetry,  PEF, 
or  FEV,.®  Treat  promptly  using  corticosteroids  or 
Beta2-agonists.  Assess  response  to  therapy  with 
objective  measures.  Discharge  patient  with  edu¬ 
cation  including  written  instructions  and  appro¬ 
priate  follow-up  plan. 

Telephone  Triage 

Assess  severity  of  exacerbation 
and  review  action  plan  with 
patient. 

Assess  the  severity  of  the  exacerbation  and  triage 
to  appropriate  provider  or  self-care  instructions. 
Review  with  the  patient  the  action  plan  devel¬ 
oped  and  actions  to  be  taken  by  the  patient. 

“  Oximetry  measures  the  oxygen  content  of  the  blood;  FEV  =  forced  expiratory  volume; 
PEE  =  Peak  expiratory  flow. 


through  application  of  the  practice  guideline,  there  should  he  ohserv- 
ahle  changes  in  use  of  primary  care  visits,  spirometry  (i.e.,  measuring 
pulmonary  capacity),  and  asthma  controller  and  rescue  medications 
for  asthma  patients  served  hy  the  MTFs,  and  there  should  he  reduc¬ 
tions  in  rates  of  use  of  emergency  rooms  and  inpatient  stays  for  exac¬ 
erbations  of  asthma. 
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As  part  of  the  process  of  developing  the  asthma  guideline,  the 
DoD/VA  working  group  responsible  for  developing  the  asthma  prac¬ 
tice  guidelines  established  a  set  of  performance  indicators  for  asthma 
care.  These  indicators  are  the  percentage  of  asthma  follow-up  visits 
with  documented  asthma  severity  level,  the  percentage  of  patients 
with  persistent  asthma  who  are  prescribed  inhaled  steroids,  the  per¬ 
centage  of  persistent  asthmatics  with  a  written  action  plan  docu¬ 
mented  in  the  past  12  months,  and  the  percentage  of  asthmatics  six 
years  or  older  with  spirometry  in  the  past  12  months. 

As  will  be  discussed  in  Chapter  Two,  we  developed  a  set  of 
hypotheses  on  effects  of  guideline  implementation  on  various  out¬ 
comes,  which  included  effects  for  the  DoD/VA  indicators.  These 
hypotheses  are  presented  in  Appendix  A.  However,  because  of  data 
limitations  and  other  constraints,  we  could  use  only  one  of  the 
DoD/VA  working  group  indicators — the  percentage  of  patients  with 
persistent  asthma  who  are  prescribed  inhaled  steroids.^ 


A  Systems  Approach  to  Implementation 

Most  studies  that  have  evaluated  the  effects  of  guideline  implementa¬ 
tion  on  health-care  practices  have  focused  rather  narrowly  on  indi¬ 
vidual  interventions  intended  to  change  provider  behavior  (e.g., 
education,  audit  and  feedback,  reminders).  Such  studies  allow 
researchers  to  design  effective  controls  with  relative  ease.  However, 
the  results  across  these  studies  have  been  quite  variable.  This  variation 
can  be  explained  in  part  by  differences  in  the  guideline  subject  mat¬ 
ters  studied,  provider  attitudes,  or  organizational  characteristics 
(Grilli  and  Lomas,  1994;  Chodoff  and  Crowley,  1995;  Lewis,  1995; 
Eastwood  and  Sheldon,  1996).  Many  of  these  studies  have  found 
weak  compliance  with  guidelines — for  example,  one  study  found  that 


^  Evaluation  of  measures  related  to  spirometry  or  oximetry  would  require  access  to  labora¬ 
tory  data  from  the  Composite  Health-Care  System  (CHCS),  which  were  not  available  for 
this  analysis.  We  also  could  not  develop  measures  for  asthma  severity  because  it  required 
extraction  of  data  from  medical  charts,  which  was  beyond  the  scope  of  this  evaluation. 
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nearly  one-third  of  the  time  primary  care  providers  fail  to  follow  even 
noncontroversial  and  evidence-hased  guideline  recommendations 
(Grol  et  ah,  1998).  The  studies  indicate  that  active  methods  of 
guideline  implementation,  such  as  physician  reminders  and  academic 
detailing  (packaged  information  on  desired  practices),  are  more 
consistently  effective  than  passive  dissemination  of  guidelines  or 
information  on  hest  practices.  A  combination  of  two  or  more  imple¬ 
mentation  approaches  has  been  shown  to  be  more  likely  to  succeed 
than  a  single  intervention,  with  multifaceted  interventions  targeted  at 
specific  barriers  being  the  most  successful  (Bero  et  ah,  1998). 

Influenced  by  systems  thinking  and  quality  improvement  meth¬ 
odologies,  health-care  managers  have  come  to  favor  multifaceted 
approaches  to  system  change,  as  opposed  to  individual  interventions, 
as  the  best  hope  for  changing  patient  care  practices  (Senge,  1990; 
Shortell,  Bennett,  and  Byck,  1998).  For  example,  the  Chronic  Care 
Model  suggests  that  improving  care  for  the  chronically  ill  requires 
major  changes  in  multiple  factors,  including  organization  and  deliv¬ 
ery  of  care,  information  systems,  doctor-patient  relationships,  patient 
self-management,  and  even  the  relationship  between  the  health  sys¬ 
tem  and  community  resources  (Wagner,  Austin,  and  Von  Korff, 
1996;  Von  Korff  et  al.,  1997).  A  premise  of  this  and  other  integrated 
models  is  that  testing  the  effects  of  single  components  of  an  interven¬ 
tion  will  yield  misleading  null  results  because  dramatic  changes  in 
outcome  occur  only  when  all  components  of  the  model  are  in  place. 

Basic  Implementation  Strategy 

The  AMEDD  has  applied  a  systems  approach  in  its  practice  guideline 
implementation  strategy.  Lessons  from  the  low  back  pain  demonstra¬ 
tion  highlighted  that  two  main  dimensions  need  to  be  addressed  in 
an  implementation  strategy  to  ensure  successful  changes  in  practices 
by  MTFs  and  other  local  facilities:  build  local  ownership,  or  “buy-in,” 
from  the  staff  responsible  for  implementing  the  new  practices  and 
ensure  that  clinical  and  administrative  systems  are  in  place  to  facili¬ 
tate  staff  adherence  to  the  guideline. 

Figure  1.2  illustrates  how  staff  buy-in  and  system  changes  inter¬ 
act  to  produce  different  implementation  results.  Having  both  local 
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ownership  and  system  support  produces  the  optimal  result,  leading  to 
likely  implementation  success.  System  support  without  local  owner¬ 
ship  produces  providers  resistant  to  implementation,  despite  clinic 
procedures  and  systems  equipped  to  support  the  process.  Provider 
ownership  without  system  support  produces  providers  who  wish  to 
change  practices  hut  are  frustrated  at  their  inability  to  overcome  har¬ 
riers  in  the  MTF  systems  that  hamper  their  ability  to  do  so.  Finally, 
with  neither  local  ownership  nor  system  support,  implementation  will 
fail. 

Six  Critical  Success  Factors  for  Implementation 

Drawing  on  published  literature  on  the  implementation  of  guidelines 
in  health-care  practice,  as  well  as  the  experiences  observed  in  the  prior 
AMEDD  demonstration  for  the  low  back  pain  guideline,  we  identi¬ 
fied  six  critical  success  factors  that  we  believe  strongly  influence  how 
successfully  an  MTF  will  be  able  to  integrate  new  practices  into  its 
clinical  and  administrative  processes.  We  used  these  factors  as  criteria 
for  evaluating  the  overall  success  of  the  asthma  guideline  implementa¬ 
tion.  The  results  of  that  evaluation  are  presented  in  Chapter  Six. 

•  Command  leadership  commitment  at  the  MTF,  regional, 
and  system  levels.  Management  leaders  at  the  MEDCOM, 


Figure  1.2 

Matrix  of  Implementation  Outcomes 

Local  ownership  No  local  ownership 


Systems  do  support 
recommended  practices 


Systems  do  not  support 
recommended  practices 


/ 

Provider  resistance 

Frustrated  providers 

X 

RANDMG3I9*I.2 
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regional,  and  MTF  levels  of  AMEDD  influence  how  front-line 
personnel  perceive  what  priority  the  system  places  on  the  use  of 
practice  guidelines.  Such  support  has  been  shown  to  he  neces¬ 
sary  to  empower  teams  to  bring  about  effective  practice  changes 
(Solberg  et  ah,  1997;  Keller,  1997;  Motwani,  Klein,  and 
Navitskas,  1999;  Savitz  and  Kaluzny,  2000). 

•  Monitoring  of  progress.  Both  the  local  MTFs  and  MEDCOM 
have  roles  in  monitoring  the  quality  of  health-care  practices 
according  to  evidence-based  standards  defined  in  practice  guide¬ 
lines  and  providing  feedback  needed  for  effective  performance 
improvement  (Palmer  and  Hargraves,  1996;  Sasala  and  Jasovsky, 
1998;  Cox,  Wilcock,  and  Young,  1999;  Eescoe-Eong  and  Eong, 
1999;  Savitz  and  Kaluzny,  2000). 

•  Guidance  and  support  to  the  MTFs  by  MEDCOM.  The 

structured  approach  and  toolkits  of  supportive  materials  pro¬ 
vided  are  resources  that  aid  the  MTFs  as  they  carry  out  actions 
to  improve  clinical  practices.  Such  support  encourages  MTFs  to 
make  needed  practice  changes  to  move  toward  consistency  in 
practices  across  Army  facilities  (Sasala  and  Jasovsky,  1998; 
Motwani,  Klein,  and  Navitskas,  1999). 

•  Guideline  champions  who  are  opinion  leaders.  Extensive  evi¬ 
dence  exists  on  the  importance  of  having  a  designated  clinical 
leader  to  serve  as  champion  for  the  practice  improvements  being 
pursued  (Palmer  and  Hargraves,  1996;  Solberg  et  ah,  1997; 
Gandhi  et  ah,  2000). 

•  Resource  support  for  champions.  To  serve  effectively  as  a 
guideline  champion,  the  designated  champion  needs  to  be  given 
adequate  dedicated  time  and  other  resource  support.  This  sup¬ 
port  also  signals  that  the  MTF  command  places  a  high  priority 
on  guideline  implementation  (Palmer  and  Hargraves,  1996; 
Eescoe-Eong  and  Eong,  1999;  Gandhi  et  al.,  2000). 

•  Institutionalization  of  new  practices.  For  sustainability,  the 
new  practices  need  to  be  integrated  into  the  standard  practices 
of  the  facility  as  quickly  as  possible  to  take  advantage  of  early 
momentum  and  achieve  early  observable  successes  on  which  fur¬ 
ther  action  can  be  built.  This  requires  successful  design  and  exe- 
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cution  of  an  action  plan  to  change  practices,  including  both 
educational  and  systems  change  interventions  (Solberg  et  ah, 
1997;  Motwani,  Klein,  and  Navitskas,  1999). 


The  AMEDD  Guideline  Implementation  Process 

The  asthma  guideline  demonstration  was  initiated  when  commanders 
at  the  demonstration  sites,  who  serve  in  a  role  equivalent  to  that  of  a 
chief  executive  officer  in  the  civilian  sector,  agreed  to  participate  in 
the  demonstration.  To  prepare  for  guideline  implementation,  the 
commander  of  each  participating  MTF  appointed  a  multidisciplinary 
implementation  team,  a  guideline  champion,  and  a  facilitator  to  help 
guide  the  MTF  team’s  implementation  activities.  The  team  consisted 
of  eight  to  ten  individuals  representing  a  mix  of  clinical  and  support 
staff  involved  in  delivering  care  for  asthma  patients.  The  champion 
was  to  be  a  primary  physician  and  an  opinion  leader  at  the  MTF  with 
a  strong  commitment  to  the  successful  implementation  of  the  guide¬ 
line.  The  facilitator  was  expected  to  have  experience  in  arranging 
group  decisionmaking  processes  as  well  as  the  ability  to  organize  work 
processes  and  to  work  with  data  for  quality  management  and  moni¬ 
toring  activities. 3 

The  systems  approach  applied  in  the  demonstrations  to  imple¬ 
ment  the  practice  guidelines  is  shown  in  Figure  1.3.  This  process  con¬ 
sisted  of  the  following  components: 

•  Practice  guideline  and  metrics.  The  official  DoD/VA  practice 
guideline  materials  are  provided  to  the  MTFs,  including  a  sum¬ 
mary  list  of  the  key  elements  of  the  guideline  and  metrics  identi¬ 
fied  by  the  guideline  expert  panel  for  monitoring  progress. 

•  Guideline  toolkit.  The  MEDCOM  and  the  Army  Center  for 
Health  Promotion  and  Preventive  Medicine  (CHPPM)  collabo¬ 
rate  in  the  development  of  a  toolkit  of  materials  to  support  the 


^  The  roles  of  the  implementation  team,  guideline  champion,  and  facilitator  are  discussed 
further  in  Chapter  Four. 
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MTFs’  guideline  implementation  activities  (e.g.,  documentation 
forms,  training  videos,  patient  education  materials,  patient 
reminder  cards). ^  Toolkits  are  provided  to  each  of  the  demon¬ 
stration  MTFs  along  with  refill  support  of  consumable  items. 

•  Kickoff  planning  conference.  The  multidisciplinary  implemen¬ 
tation  teams  participate  in  a  two-day  interactive  meeting  to 
develop  guideline  implementation  strategies  and  action  plans. 
Members  of  the  teams  interact  with  each  other  and  with  RAND 
and  MEDCOM  facilitators. 

•  MTF  implementation  activities.  Following  the  kickoff  confer¬ 
ence,  MTF  implementation  teams  carry  out  their  action  plans. 
They  prepare  monthly  reports  that  summarize  recent  activities, 
successes,  challenges,  and  aid  needed  to  support  their  work. 

•  Information  exchange.  Teams  are  encouraged  to  share  experi¬ 
ences  with  each  other  and  the  RAND  and  MEDCOM  facilita¬ 
tors  to  learn  from  errors  and  build  on  successes.  Communica- 


Figure  1.3 

Guideline  Implementation  Process 


RAND  MG319-1. 3 


^  CHPPM  performs  many  of  the  Army’s  public  health  and  preventive  health  activities  for 
both  wartime  and  domestic  aspects  of  the  Army’s  activities. 
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tion  occurs  through  MEDCOM-supported  listservs  (mailing  list 
program  for  communicating  with  other  people  who  have  suh- 
scrihed  to  the  same  list)  and  direct  e-mail  communication. 

•  Monitoring  of  progress.  Monitoring  of  implementation  pro¬ 
gress  is  performed  hy  hoth  MEDCOM  and  the  implementation 
teams  at  the  participating  MTFs  through  site  visits  and  data 
monitoring.  Monitoring  at  the  MEDCOM  level  focuses  on 
metrics  developed  in  the  DoD/VA  guideline  process.  MTFs  are 
expected  to  use  the  DoD/VA  metrics  in  their  monitoring  when¬ 
ever  data  availability  permits.  They  also  are  encouraged  to  track 
additional  measures  that  provide  information  relevant  to  their 
specific  practice  improvement  priorities  and  practices. 


The  Demonstration  Sites 

Four  MTFs  in  the  Southeast  Regional  Medical  Command  served  as 
demonstration  sites  for  implementation  of  the  asthma  guideline: 

•  Blanchfield  Army  Community  Hospital  (ACH)  at  Fort  Camp- 
hell, 

•  Eisenhower  Army  Medical  Center  (AMC)  at  Fort  Gordon, 

•  Martin  ACH  at  Fort  Benning,  and 

•  Moncrief  ACH  at  Fort  Jackson. 

The  four  MTFs  participating  in  the  demonstration  represented 
diverse  patient  populations,  facility  sizes,  and  service  mixes.  Martin, 
Blanchfield,  and  Moncrief  are  ACHs  that  provide  mainly  primary 
care  services  with  some  specialty  care.  Dwight  D.  Eisenhower  AMC 
provides  primary,  secondary,  and  tertiary  care  and  trains  AMEDD 
clinical  personnel.  The  types  of  patients  served  and  service  activity  of 
the  MTFs  are  shown  in  Table  1.2. 

The  patient  populations  served  by  Blanchfield  ACH  are  primar¬ 
ily  active-duty  personnel  and  dependents,  whereas  Moncrief  ACH 
serves  a  relatively  large  retiree  population,  as  does  Martin  ACH  to 
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Table  1.2 

Profiles  of  the  MTFs  Participating  in  the  Asthma  Guideline  Demonstration 


Eisenhower 

AMC 

Martin 

ACH 

Blanchfield 

ACH 

Moncrief 

ACH 

DMIS  Number 

47 

48 

60 

105 

Number  of  beneficiaries 

Active-duty 

10,432 

15,463 

24,382 

9,318 

Active-duty  dependents 

15,492 

21,191 

31,981 

9,658 

Retirees,  dependents, 

and  survivors 

31,072 

32,135 

23,897 

28,955 

Other 

764 

869 

389 

3,532 

Total 

57,760 

69,658 

80,649 

51,463 

Percentage  retirees 

53.8 

46.1 

29.6 

56.2 

Service  activity 

Inpatient  dispositions* 

1,932 

2,587 

3,281 

1,357 

Same-day  surgeries* 

2,952 

888 

2,412 

503 

Outpatient  visits* 

239,684 

226,090 

293,530 

123,074 

NOTE:  All  data  are  from  the  Corporate  Executive  Information  System  (CEIS).  Aster¬ 
isked  items  are  for  the  period  October  1998-l\/larch  1999.  All  other  data  are  from  FY 
1998.  DMIS  =  Deployment  Medication  Information  Sheet. 


a  lesser  extent.  Eisenhower  AMC  also  serves  a  large  retiree  popula¬ 
tion.  These  population  differences  are  reflected  in  the  MTFs’  ratios  of 
retirees  to  active-duty  personnel,  which  range  from  a  low  of  0.98  at 
Blanchfield  ACH  to  a  high  of  3.1 1  at  Moncrief  ACH. 

The  four  MTFs  also  vary  in  other  clinical  and  educational  activi¬ 
ties,  as  shown  in  Table  1.3.  Martin  ACH  and  Eisenhower  AMC  pro¬ 
vide  extensive  medical  education  training.  Martin  ACH  operates  the 
Army’s  oldest  Family  Practice  Residency  Program  and  the  clinical 
portion  of  the  Physicians  Assistants  Program.  Eisenhower  AMC  and 
Blanchfield  ACH  have  asthma  resource/education  centers. 
Eisenhower  AMC  also  has  a  health  and  wellness  center  that  serves  as  a 
source  for  health  education  classes  and  screenings.  All  the  MTFs 
except  Moncrief  ACH  have  allergy  clinics.  Martin  ACH  and  Blanch- 
field  ACH  have  numerous  primary  care/family  practice  clinics  and 
troop  medical  clinics  (TMCs).  Sites  vary  widely  in  their  access  to  and 
sophistication  of  computer  support. 

To  preserve  the  confidentiality  of  participating  MTFs,  these 
facilities  will  he  designated  anonymously  in  this  report  as  Demonstra¬ 
tion  Sites  One  through  Four. 
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Table  1.3 

Activities  of  the  MTFs  Participating  in  the  Asthma  Guideline  Demonstration 


Activity 

Eisenhower 

AMC 

Martin 

ACH 

Blanchfield 

ACH 

Moncrief 

ACH 

Medical  education 

X 

X 

Asthma  resource  center 

X 

X 

Wellness  center 

X 

Allergy  clinics 

X 

X 

X 

Primary  care  clinics 

X 

X 

X 

X 

Troop  medical  clinics 

X 

X 

The  RAND  Evaluation 

RAND’s  evaluation  of  the  asthma  demonstration  consisted  of  two 
components:  a  process  evaluation  and  an  analysis  of  the  effects  of  the 
guideline  on  a  series  of  outcome  measures.  The  first  component  was 
an  evaluation  of  the  implementation  process  at  the  four  participating 
MTFs.  The  second  component  was  an  analysis  of  the  effects  of  the 
guideline  on  service  utilization  at  the  four  MTFs.  Baseline  service 
utilization  (prior  to  guideline  implementation)  was  also  assessed  to 
establish  a  benchmark  for  current  practice.  Service  utilization  was  also 
assessed  at  six  control  sites  to  rule  out  the  effects  of  temporal  trends. 
Included  in  the  analysis  of  service  utilization  was  an  assessment  of  the 
adequacy  of  Army  medical  databases  for  monitoring  the  results  of  the 
guideline  implementation  as  well  as  future  follow-up  and  provider 
feedback.  The  impact  of  the  guideline  on  costs  was  also  assessed.  A 
full  description  of  the  methods  used  in  the  RAND  evaluation  can  be 
found  in  Chapter  Two. 


Organization  of  This  Report 

In  the  remainder  of  the  report,  we  present  our  evaluation  methods 
and  findings.  Chapter  Two  describes  the  methods  and  data  used  for 
the  evaluation.  Chapter  Three  provides  information  on  the  size  and 
characteristics  of  the  asthma  population  served  by  Army  MTFs  and 
profiles  baseline  performance  for  the  ten  MTFs  included  in  the 
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evaluation  on  each  of  the  measures  used  to  assess  the  effects  of  the 
guideline  on  clinical  practices  for  asthma  patients.  Results  of  the 
process  evaluation  are  reported  in  Chapter  Four,  and  results  of  the 
evaluation  of  guideline  effects  are  presented  in  Chapter  Five.  In 
Chapter  Six,  we  synthesize  the  results  of  the  full  evaluation  and  iden¬ 
tify  lessons  learned  and  implications  for  systemwide  guideline  imple¬ 
mentation  strategies  and  include  our  recommendations. 


CHAPTER  TWO 

Methods  and  Data 


The  RAND  evaluation  for  the  asthma  guideline  demonstration  gath¬ 
ered  information  about  both  the  processes  of  implementing  the 
practice  guideline  at  participating  MTFs  and  the  effects  of  these 
implementation  activities  on  delivery  of  care  for  asthma  patients.  In 
this  chapter,  we  summarize  the  methods  and  data  for  these  two  eval¬ 
uation  components.  Additional  details  about  methodology  are  pro¬ 
vided  in  Appendix  B. 

Implementation  of  a  clinical  practice  guideline  is  one  type  of 
quality  improvement  intervention.  An  evaluation  of  any  quality 
improvement  intervention  should  recognize  the  incremental  nature  of 
these  processes,  which  require  time  to  achieve  lasting  improvements. 
A  comprehensive  evaluation  of  guideline  implementation,  therefore, 
would  encompass  the  following  three  phases  of  emphasis: 

•  Introducing  new  practices.  Evaluating  initial  practice  imple¬ 
mentation  efforts  emphasizes  documentation  of  the  extent  to 
which  effective  action  plans  are  developed  and  the  intended 
actions  are  implemented.  Process  evaluation  methods  are  used 
and  sometimes  feedback  is  provided  to  participants  early  in  the 
process  to  help  them  strengthen  their  interventions. 

•  Achieving  intended  changes  in  practices.  Evaluating  process 
change  involves  monitoring  short-term  effects  on  service- 
delivery  methods  and  activity  through  use  of  relevant  quantifi¬ 
able  outcome  measures. 


17 
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•  Improving  patient  outcomes.  Assessing  patient  outcomes  is  a 
longer-term  effort  that  also  uses  relevant  quantifiable  outcome 
measures.  Patient  outcomes  were  not  assessed  in  this  study. 

In  the  second  phase  of  the  assessment  (evaluating  changes  in 
practices),  the  data  collected  by  RAND  were  also  used  to  assess  the 
availability  and  usability  of  DoD  administrative  data  for  monitoring 
effects  of  practice  improvement  processes  on  clinical  practices.  In  this 
analysis,  we  assessed  which  indicators  could  be  measured  using  cen¬ 
trally  available  (administrative)  data  and  which  indicators  required 
data  available  only  at  the  MTFs.  We  also  examined  coding  and  mea¬ 
surement  issues  that  must  be  addressed  to  establish  valid  measures  of 
the  indicators  using  the  administrative  data.  We  document  our  find¬ 
ings  on  these  issues  in  Chapter  Five. 


Process  Evaluation  Methods 

To  learn  from  the  experience  of  the  MTFs  participating  in  the  dem¬ 
onstration,  the  RAND  team  used  a  participant-observer  approach, 
which  provides  for  regular  interactions  between  the  evaluator  and  the 
teams  being  observed  in  a  process  evaluation.  This  approach  allows 
for  exchange  of  information  and  facilitates  shared  learning  with  the 
MTFs  throughout  the  demonstration  and  evaluation  process.  The 
purposes  of  the  implementation  process  evaluation  were  to  accom¬ 
plish  the  following: 

•  Document  the  actions  and  experiences  of  the  demonstration 
MTFs  with  practice  guideline  implementation  and  assess  per¬ 
formance  relative  to  each  of  the  six  key  success  factors  described 
in  Chapter  One. 

•  Identify  areas  where  the  policies,  systems,  and  processes  estab¬ 
lished  by  the  AMEDD  can  be  strengthened  to  better  support 
systemwide  guideline  implementation. 
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To  understand  the  full  dynamics  of  the  practice  guideline 
implementation,  we  gathered  information  on  the  interactions  of  the 
many  aspects  of  the  system  and  from  many  stakeholders,  including 
the  implementation  team,  treatment  program  leadership,  middle 
management,  clinical  and  administrative  staff,  and  patients. 

Information  was  collected  through  a  climate  survey  conducted 
during  the  kickoff  conference,  during  site  visits,  and  from  monthly 
progress  reports  prepared  hy  the  team  facilitators  of  the  participating 
MTFs  and  submitted  to  RAND.  Throughout  the  study,  we  main¬ 
tained  an  ongoing  communication  process  to  provide  a  structure 
through  which  implementing  MTFs  could  get  assistance  from  each 
other,  MEDCOM,  or  RAND. 

We  briefly  describe  each  of  these  methods  here.  Additional 
details  are  presented  in  Appendix  B. 

Climate  Survey 

At  the  kickoff  conference,  members  of  the  MTF  command  and 
implementation  teams  were  asked  to  complete  a  written  question¬ 
naire  designed  to  measure  motivation  and  attitudes  toward  quality 
improvement  and  the  practice  guidelines  specifically.  The  climate 
survey  consisted  of  four  modules  that  addressed  motivation  for 
guideline  implementation,  supportiveness  of  climate,  attitudes  toward 
practice  guidelines,  and  efforts  to  improve  quality  of  care.  These 
modules  are  provided  in  Appendix  C. 

Each  module  contained  sets  of  items  with  scaled  ratings  options, 
the  responses  to  which  were  summarized  to  obtain  overall  scores  for 
each  climate  component.  Eor  the  module  on  motivation,  respondents 
were  asked  to  rate  the  importance  of  each  of  eight  quality  improve¬ 
ment  factors.  They  also  were  asked  to  rate  the  current  status  of  their 
MTE  on  each  factor.  The  modules  on  climate  for  practice  guideline 
implementation  and  attitudes  toward  guidelines  contained  seven  and 
six  items,  respectively. 

The  climate  survey  results  were  used  to  assess  the  readiness  of 
the  participants  and  their  organizations  to  embark  on  implementa¬ 
tion  of  practice  guidelines  and  other  evidence-based  practices.  The 
survey  results  are  reported  in  Chapter  Eour. 
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Evaluation  Site  Visits 

Two  evaluation  site  visits  were  conducted  at  each  demonstration  site 
as  they  were  implementing  the  asthma  practice  guideline:  one  sched¬ 
uled  for  the  third  month  after  the  MTFs  began  implementing  their 
action  plans  and  a  second  approximately  six  months  after  the  first 
visits.*  Two  of  the  sites  started  implementation  hy  November  and  the 
other  two  sites  had  begun  activities  by  early  January  2000.^  The 
demonstration  and  evaluation  schedule  was  as  follows: 

•  September  1999:  Kickoff  planning  meeting  for  MTFs. 

•  February  and  March  2000:  First  evaluation  site  visits  three 
months  after  the  MTFs  began  implementing  their  plans  (Febru¬ 
ary  for  the  two  MTFs  that  started  implementation  in  November 
and  early  March  for  the  two  MTFs  that  started  in  January). 

•  September  2000:  Second  evaluation  site  visits  six  months  after 
the  first  evaluation  site  visits  (all  four  MTFs). 

In  preparation  for  the  site  visits,  RAND  developed  an  agenda 
for  the  group  meetings,  individual  interviews,  and  focus  groups  that 
we  wanted  to  perform.  The  facilitator  of  each  implementation  team 
worked  with  that  agenda  to  schedule  the  meetings  with  implementa¬ 
tion  team  members  and  other  individuals  involved  in  the  implemen¬ 
tation  process. 

MEDCOM  staff  participated  in  the  site  visits  with  the  RAND 
evaluation  team,  which  allowed  MEDCOM  to  learn  directly  from 
the  MTEs’  experiences.  RAND  conducted  the  interviews  and  focus 
groups  and  the  MEDCOM  staff  provided  technical  assistance  and 
other  support  to  the  MTE  teams  during  the  site  visit.  MEDCOM 
staff  were  also  present  as  observers  at  interviews  and  focus  groups  to 


*  We  also  conducted  an  introductory  visit  to  each  site  a  few  months  before  the  September 
1999  kickoff  meeting,  during  which  we  got  acquainted  with  the  MTF  implementation  teams 
and  the  facilities  in  which  they  work. 

^  Following  the  asthma  guideline  kickoff  conference  in  August  1999,  the  delays  MTFs  expe¬ 
rienced  in  starting  implementation  actions  stemmed  from  time  conflicts  through  the  holi¬ 
days  as  well  as  some  startup  inefficiencies. 
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learn  firsthand  from  the  MTFs’  experiences  and  feedback.  At  the 
conclusion  of  each  evaluation  visit,  we  briefed  the  MTF  command 
group  about  what  we  had  learned  and  what  issues  we  identified, 
which  is  a  standard  step  for  site  visit  processes  at  military  facilities. 

During  the  postimplementation  site  visits,  we  interviewed  the 
guideline  champion,  team  facilitator,  and  implementation  team 
members  to  learn  their  respective  perspectives  on  the  process  and 
their  experiences.  Semistructured  interview  methods  were  used  for  all 
interviews  and  group  discussions,  working  from  predefined  lists  of 
questions  to  cover  during  each  session.  All  individuals  were  informed 
that  participation  in  the  interviews  and  focus  groups  was  voluntary 
and  that,  to  protect  their  privacy,  everything  we  reported  from  the 
evaluation  would  be  structured  to  shield  the  identity  of  individuals 
who  were  sources  of  comments  or  observations. 

Separate  focus  groups  were  conducted  with  each  of  three  types 
of  stakeholder:  the  implementation  team  members,  providers,  and 
other  clinic  staff.  The  MTF  champions  and  facilitators  invited  all 
individuals  in  each  stakeholder  group  to  participate  in  the  focus 
groups.  Attendance  by  those  invited  generally  was  high,  so  we  had 
confidence  that  we  obtained  feedback  representative  of  each  group. 
To  give  focus  group  participants  the  privacy  to  express  their  opinions 
freely,  the  implementation  team  did  not  attend  the  focus  groups  for 
providers  or  other  clinic  staff.  At  the  start  of  each  focus  group,  we 
advised  the  participants  of  the  informed  consent  provisions.  We  also 
described  how  this  focus  group  fit  into  the  overall  site  visit  process, 
including  the  final  briefing  to  the  MTF  command. 

Using  a  written  protocol,  we  asked  participants  in  each  focus 
group  questions  regarding  how  they  felt  about  the  guideline  imple¬ 
mentation,  how  they  worked  with  the  practice  guideline,  how  they 
were  affected  by  the  implementation  process,  and  what  issues  or  con¬ 
cerns  they  had  about  the  process. 

Finally,  we  interviewed  the  command  leadership  of  the  MTF. 
These  sessions  allowed  us  to  communicate  to  the  leadership  both 
progress  being  made  by  their  implementation  teams  and  issues 
regarding  administrative  or  clinical  barriers  hampering  guideline 
implementation.  We  also  used  these  sessions  to  obtain  feedback  from 
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the  MTF  leadership  on  the  practice  guideline,  the  implementation 
process,  and  the  level  of  priority  they  placed  on  this  work. 

At  the  second  postimplementation  site  visits  (at  month  ten  of 
implementation),  we  began  each  interview  or  focus  group  hy  asking 
the  participants  to  complete  a  written  questionnaire  that  gathered 
information  on  individuals’  views  of  the  guideline  and  experiences  in 
the  implementation  process.  Separate  questionnaires  were  used  for 
each  of  three  groups:  providers,  other  clinic  staff,  and  the  implemen¬ 
tation  team  members.  The  provider  questionnaire  can  be  found  in 
Appendix  D  as  an  example.^  Because  the  number  of  respondents  was 
small  for  each  group,  we  could  not  analyze  or  interpret  the  results  sta¬ 
tistically.  However,  the  survey  information  helped  identify  issues  and 
experiences  useful  to  the  process  evaluation. 

Monthly  Reports 

The  final  source  of  process  evaluation  information  was  monthly  pro¬ 
gress  reports  prepared  by  the  team  facilitators  at  the  participating 
MTFs  and  submitted  to  RAND.  These  reports  obtained  information 
on  the  current  status  of  implementation  actions  relative  to  the 
planned  action  schedule,  successes  achieved,  challenges  being  faced  by 
the  teams,  management  methods  for  those  challenges,  and  additional 
assistance  that  could  be  provided  by  MEDCOM.  These  reports  doc¬ 
umented  the  evolution  of  the  MTFs’  implementation  strategy  and 
progress.  We  used  this  information  in  our  development  of  interview 
and  focus  group  protocols  for  the  second  postimplementation  site 
visits.  The  reports  also  stimulated  follow-up  action  by  both  the  MTFs 
and  MEDCOM  as  the  MTEs  identified  issues  requiring  resolution. 


Outcome  Evaluation 

The  outcome  analysis  had  four  goals: 


^  Copies  of  the  other  questionnaires  may  be  obtained  from  the  authors. 
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•  Document  the  changes  in  clinical  process  and  service  activity  in 
the  clinics  that  implement  the  asthma  practice  guideline. 

•  Document  changes  in  asthma  patients’  service  utilization  attrih- 
utahle  to  the  clinical  process  changes  that  have  occurred. 

•  Assess  average  MTF  costs  of  care  for  treatment  of  the  asthma 
patients  they  serve  and  determine  how  those  costs  may  have 
been  affected  hy  implementation  of  the  guideline. 

•  Examine  the  usefulness  of  the  metrics  and  measurement  meth¬ 
ods  used  in  the  demonstration  and  implications  for  how  best  to 
establish  an  effective  system  for  routine  monitoring  of  progress. 

The  importance  of  the  fourth  goal  cannot  be  overstated.  A 
viable  monitoring  process,  including  well-chosen,  relevant  measures, 
is  essential  for  an  MTF  to  retain  the  gains  it  achieved  by  modifying 
practices  as  recommended  by  the  guideline.  This  feedback  loop  con¬ 
tinues  to  provide  MTF  staff  with  program  quality  information,  and  it 
maintains  the  visibility  of  the  measures  being  reported  as  priorities  for 
high-quality  performance. 

Hypotheses  for  Effects  of  Implementation  of  the  Asthma  Guideline 

Our  evaluation  of  the  effects  of  the  asthma  guideline  at  the  demon¬ 
stration  sites  was  guided  by  several  hypotheses  that  addressed  possible 
practice  changes  and  clinical  measures  for  asthma  patients.  These 
hypotheses  were  informed  by  the  official  DoD/VA  asthma  metrics  as 
well  as  by  other  aspects  of  care."*  Reflecting  the  DoD/VA  practice 
guideline  and  metrics,  the  hypotheses  addressed  the  initial  assessment 
of  patients,  asthma  management  and  follow-up,  and  management  of 
exacerbations. 

We  took  the  approach  of  identifying  a  broad  scope  of  possible 
outcomes.  This  was  done  to  inform  priority-setting  for  future  quality 
and  performance  monitoring  activities  and  to  stimulate  improve¬ 
ments  in  data  capabilities  to  measure  important  indicators  of  quality 
care.  The  hypotheses  that  generated  our  outcome  measures  are  listed 


^  As  noted  in  Chapter  One,  because  of  limitations  of  available  DoD  data,  we  could  not  use 
all  of  the  DoD/VA  indicators. 
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in  Table  2.1.  The  outcome  measures  themselves  are  discussed  later  in 
this  chapter. 

Evaluation  Design 

To  test  these  hypotheses,  we  used  an  interrupted  time  series  control- 
group  design.  Trends  for  the  outcome  measures  were  estimated  for 
one  year  before  the  demonstration  sites  began  using  the  asthma 
guideline  and  for  one  year  after  its  start  date.  This  approach  allowed 
us  to  estimate  annual  utilization  measures,  and  it  also  controlled  for 
any  seasonality  effects  on  asthma  care.  A  control  group  of  six  addi¬ 
tional  MTFs  was  used  to  adjust  for  underlying  historical  trends  that 
otherwise  could  threaten  the  validity  of  findings  at  the  demonstration 
MTFs.  The  evaluation  design  is  shown  graphically  in  Figure  2.1. 

Administrative  data  on  utilization,  medications,  and  laboratory 
tests  were  collected  for  asthma  patients  enrolled  in  TRICARE  Prime 
at  each  of  the  four  demonstration  MTFs  or  six  control  MTFs 
included  in  the  analysis.  Data  for  each  year  in  the  study  period  were 
broken  into  four  three-month  quarters  so  we  could  track  trends  over 
time  within  the  two-year  study  period. 

Choice  of  Demonstration  and  Control  Sites 

As  described  in  Chapter  One,  four  asthma  demonstration  sites  were 
selected  in  the  Southeast  Region.  MEDCOM  and  the  Army  Patient 


Table  2.1 

Expected  Effects  of  Proactive  Asthma  Care  Management 


Increase  in  percentage  of  asthma  patients  using  long-term  controllers  (inhaled 
corticosteroid,  leukotriene  inhibitor,  Betaj  agonist/CS,  or  oral  corticosteroid) 

Increase  in  percentage  of  asthma  patients  using  complementary  maintenance  medi¬ 
cations  (Betaj  agonist/LA  or  methyixanthine) 

Increase  in  percentage  of  asthma  patients  using  short-acting  rescue  medications 
(Betaj  agonist) 

Decrease  in  asthma-related  outpatient  visit  rate  per  1,000  asthma  patients 
Decrease  in  asthma-related  emergency  room  visit  rate  per  1,000  asthma  patients 
Decrease  in  asthma-related  hospitalization  rate  per  1,000  asthma  patients 
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Figure  2.1 

Evaluation  Timeline 
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Administration  Systems  and  Biostatistical  Activity  (PASBA)  also 
selected  six  control  MTFs  that  matched  reasonably  well  to  the  dem¬ 
onstration  sites  based  on  peer  groupings  already  established  for 
benchmarking  within  AMEDD  and  that  did  not  undertake  initiatives 
to  improve  care  for  asthma  patients  during  the  two-year  study  period. 
The  peer  groupings  developed  by  PASBA  were  used  to  identify  con¬ 
trol  MTFs  that  were  similar  to  the  demonstration  MTFs  in  terms  of 
size  and  service  mix. 

The  Asthma  Population 

The  analysis  in  this  evaluation  works  with  three  distinct  asthma 
populations: 

•  Total  asthma  population — ^All  TRICARE-eligible  patients  with 
asthma  who  during  the  two  study  years  defined  for  the  evalua¬ 
tion  either  received  asthma  care  or  prescription  for  asthma 
medication  at  least  once  at  any  Army  MTF  in  the  continental 
United  States  or  resided  in  an  Army  MTF  catchment  area  and 
received  asthma  care  from  a  network  provider  or  filled  an 
asthma  medication  prescription  through  the  National  Mail 
Order  Pharmacy  (NMOP). 
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•  MTF  users — Patients  in  the  total  asthma  population  who 
received  inpatient  or  outpatient  asthma  care  at  least  once  during 
the  study  years  from  any  of  the  MTFs  that  were  demonstration 
or  control  sites. 

•  MTF  enrollees — Patients  in  the  total  asthma  population  who 
during  the  study  period  were  continuously  enrolled  in 
TRICARE  Prime  at  one  of  the  MTFs  that  were  demonstration 
or  control  sites. 

Defining  the  Total  Population.  We  examine  the  total  population 
to  better  understand  the  mix  of  asthma  patients  served  hy  the  Army 
MTFs  and  to  characterize  how  these  patients  use  MTFs  or  other 
community-hased  providers  for  their  health-care  services.  The  total 
asthma  population  was  identified  based  on  information  available  in 
the  service  utilization  data  for  MTF  care  or  network  provider  care 
and  in  records  of  prescription  medications  filled  by  either  MTF 
pharmacies  or  the  NMOP  program.  A  patient  was  identified  for  the 
total  asthma  population  if  at  least  one  record  of  any  of  the  following 
types  was  found  for  the  period  between  January  1999  and  March 
2001: 

•  MTF  encounter  record  with  an  International  Classification  ofi 
Diseases,  Ninth  Revision  (ICD-9),  diagnosis  code  for  asthma 
(code  493)  in  any  of  the  diagnosis  code  positions  and  care  pro¬ 
vided  in  an  Army  facility  (all  Army  facility  IDs,  including 
MTFs,  clinics,  and  TMCs); 

•  Network  provider  claim  with  an  ICD-9  diagnosis  code  for 
asthma  (code  493)  or  in  any  of  the  diagnosis  code  positions  and 
patient  resided  in  the  catchment  area  of  an  Army  MTF  or  health 
center  (parent  facility  IDs  were  used  to  pick  up  those  residing  in 
the  parent  facility  catchment  area  even  if  they  used  a 
freestanding  clinic  with  a  separate  ID); 

•  At  least  two  prescriptions  filled  at  an  Army  MTF  pharmacy  that 
was  on  a  defined  list  of  asthma  medications  (see  Appendix  B);  or 

•  At  least  two  prescriptions  filled  through  the  NMOP  program  on 
a  defined  list  of  asthma  medications  (see  Appendix  B)  and  the 
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patient  Zip  code  of  residence  was  within  the  catchment  area  of 
one  of  the  Army  MTFs  or  health  centers. 

We  identified  a  total  of  121,465  patients  in  the  total  asthma 
population  during  the  first  year  of  this  study  and  120,980  patients 
during  the  second  study  year.  Additional  details  on  the  composition 
and  demographics  of  these  populations  are  provided  in  Chapter 
Three  (Tables  3.1  and  3.2). 

Defining  MTF  Users  and  Enrollees.  These  two  suhpopulations  of 
asthma  patients  were  identified  for  our  analysis  of  the  effects  of  the 
demonstration.  The  MTF  users  were  identified  as  any  patients  who 
had  at  least  one  encounter  (inpatient  or  outpatient)  for  asthma  care 
services  at  one  of  the  study  MTFs  at  any  time  during  the  study  year. 
The  MTF  enrollees  were  identified  as  those  who  were  reported  in  all 
encounter  or  claims  records  for  a  study  year  as  being  enrolled 
TRICARE  Prime  at  one  of  the  demonstration  or  control  sites. 

The  sample  sizes  at  the  demonstration  and  control  MTFs  for  the 
analysis  of  the  guideline  effects  are  presented  in  Table  2.2.  For  study 
year  one,  a  total  of  8,439  asthma  patients  used  one  of  the  demonstra¬ 
tion  sites  and  14,177  patients  used  one  of  the  control  sites  at  least 
once  during  the  year  (MTF  users). 

Our  study  sample  for  the  effects  analysis  consisted  only  of  the 
MTF  enrollees.  For  the  first  year,  the  sample  included  4,631  patients 


Table  2.2 

Asthma  Patient  Sample  Sizes  for  the  Demonstration  and  Control  MTFs, 
by  Study  Year 


Study  Year  One 

Study  Year  Two 

Enrollee  Group 

Demonstration 

Sites 

Control 

Sites 

Demonstration 

Sites 

Control 

Sites 

Study  MTF  users 

Study  MTF  enrollees 

8,439 

14,177 

9,057 

13,742 

(study  sample) 

4,631 

8,339 

4,289 

7,031 

Other  users 

3,808 

5,838 

4,768 

6,711 

NOTE:  MTF  user  refers  to  asthma  patients  who  used  an  MTF  for  asthma  care  during 
the  year;  MTF  enrollee  refers  to  asthma  patients  enrolled  in  TRICARE  Prime  with  a 
PCM  based  at  an  MTF. 
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enrolled  at  one  of  the  four  demonstration  MTFs  and  8,339  enrolled 
at  one  of  the  five  control  MTFs.  For  the  second  year,  the  sample 
included  4,289  patients  enrolled  at  the  demonstration  sites  and  7,031 
enrolled  at  the  control  sites. 

Data  Sources 

The  analyses  conducted  in  this  study  required  data  on  outpatient  vis¬ 
its,  hospital  inpatient  stays,  use  of  medications  for  asthma  manage¬ 
ment,  and  laboratory  tests.  Table  2.3  shows  the  sources  of  the  data. 

All  of  these  data  except  the  Medical  Expense  and  Performance 
Reporting  System  (MEPRS)  data  were  extracted  by  PASBA,  and  the 
extracted  data  files  were  transmitted  to  RAND  for  analysis.  Unit  cost 
estimates  based  on  the  MEPRS  data  had  been  obtained  by  RAND 
directly  from  DoD  as  part  of  its  evaluation  of  the  Medicare-DoD 
Subvention  Demonstration.  Details  of  the  methods  for  extracting 
data  from  these  sources  and  for  construction  of  the  analysis  files  are 
presented  in  Appendix  B. 


Table  2.3 

Source  of  Data  for  Analyses 


Data  Type 

Source 

MTF  outpatient  visits 

Standard  Ambulatory  Data  Record  (SADR)  data 
base  extracted  from  MTF  Ambulatory  Data  Sys¬ 
tem  (ADS)  data 

MTF  inpatient  stays 

Standard  Inpatient  Data  Record  (SIDR)  data  base 
extracted  from  MTF  Composite  Health-Care  Sys¬ 
tem  (CHCS)  data 

Outpatient  visits  and  inpatient 

Health-Care  Service  Records  (HCSRs)  maintained  in 

stays  for  network  providers 

the  TRICARE  Civilian  Health  and  Medical  Program 
of  the  Uniformed  Services  (CHAMPUS)  data  sys¬ 
tem 

MTF  pharmacy  data 

Uniformed  Services  Prescription  Database  (USPD) 
maintained  by  the  Pharmacoeconomic  Center 
(PEC) 

NMOP  data 

TRICARE  NMOP  data  system 

Financial  data 

Medical  Expense  and  Performance  Reporting  Sys¬ 
tem  (MEPRS)= 

“  The  DoD  financial  management  data  system  that  maintains  facility-level  financial 
data  for  all  MTFs.  This  system  uses  a  standard  book  of  accounts  to  maintain  records  of 
operating  costs,  staff  time  and  costs,  and  units  of  activities  for  each  cost  center. 
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Identification  of  the  total  asthma  population  using  the  MTFs 
included  only  records  for  which  asthma  had  been  included  as  a  diag¬ 
nosis  code.  Once  we  identified  the  MTF  enrollees  and  MTF  users,  we 
also  examined  data  for  these  individuals  on  all  MTF  services,  network 
provider  services,  and  pharmacy  use  for  all  patients  in  these  groups. 

Outcome  Measures 

The  definitions  of  the  numerators  and  denominators  used  to  calculate 
the  asthma  practice  guideline  outcome  measures  are  listed  in  Table 
2.4.  These  measures  operationalize  the  hypotheses  presented  in  Table 

2.1. 

Definition  of  Key  Variables 

Variables  for  enrollment  status,  health-care  service  use,  and  medica¬ 
tions  were  derived  for  calculation  of  the  indicators  being  analyzed. 
We  also  defined  variables  for  the  military  status,  gender,  and  age  of 
each  patient  in  our  study  sample.  Definitions  of  these  variables  with 
coding  details  are  given  in  Appendix  B. 

Analytic  Methods 

The  first  step  in  the  analysis  was  to  calculate  each  indicator  for  epi¬ 
sodes  in  each  quarter-year  of  the  study  period.  For  each  measure,  we 
then  estimated  the  baseline  performance  for  the  MTFs,  described 
quarterly  trends  for  the  demonstration  and  control  sites,  and  tested 
the  statistical  significance  of  any  observed  differences  in  performance 
of  the  demonstration  site  compared  to  the  control  sites.  See  Appendix 
B  for  details  on  the  statistical  tests. 

Benchmarking.  We  combined  data  from  the  baseline  study  year 
of  CY  1999  to  create  baseline  measures  on  the  six  indicators  for  the 
ten  MTFs  included  in  the  study  as  either  demonstration  or  control 
sites.  For  each  indicator,  we  compared  the  performance  of  each  MTF 
to  the  mean  performance  of  all  other  MTFs  combined  (i.e.,  excluding 
the  index  MTF).  We  did  not  adjust  for  multiple  comparisons,  which 
can  increase  the  probability  of  Type  1  errors  (false  negatives),  but  we 
report  significance  levels  at  both  the  0.05  and  0.01  thresholds.  An 


30  Implementation  of  the  Asthma  Practice  Guideline  in  AMEDD 


Table  2.4 

Indicators  Used  to  Measure  Effects  on  Clinical  Practices  Related  to 
Implementation  of  the  DoD/VA  Asthma  Practice  Guideline 


Calculation  of  the  Indicator 

Indicator 

Numerator 

Denominator 

Percentage  of  asthma 
patients  prescribed  long¬ 
term  controllers 

Number  of  asthma  patients  in 
the  denominator  who  are 
prescribed  long-term  con¬ 
trollers® 

Number  of  patients 
with  asthma  in  the 
study  year 

Percentage  of  asthma 
patients  prescribed  com¬ 
plementary  control  medi¬ 
cations 

Number  of  asthma  patients  in 
the  denominator  who  are 
prescribed  complementary 
medications® 

Number  of  patients 
with  asthma  in  the 
study  year 

Percentage  of  asthma 
patients  prescribed  a 
short-acting  rescue  medi¬ 
cation 

Number  of  patients  in  the 
denominator  who  are  pre¬ 
scribed  a  rescue  medication® 

Number  of  patients 
with  asthma  in  the 
study  year 

Rate  of  asthma-related 
outpatient  visits  per  1,000 
asthma  patients 

Rate  of  asthma-related 
emergency  room  visits 
per  1,000  asthma  patients 

Rate  of  asthma-related 
hospitalizations  per  1,000 
asthma  patients 

Number  of  asthma-related 
outpatient  visits  for  patients 
in  the  denominator 

Number  of  asthma-related 
emergency  room  visits  for 
patients  in  the  denominator 

Number  of  asthma-related 
hospitalizations  for  patients 
in  the  denominator 

Number  of  patients 
with  asthma  in  the 
study  year 

Number  of  patients 
with  asthma  in  the 
study  year 

Number  of  patients 
with  asthma  in  the 
study  year 

^  Long-term  controller  medications  included  inhaled  corticosteroid,  leukotriene  inhibi¬ 
tor,  Beta2-agonist/CS,  and  oral  corticosteroid.  Complementary  medications  included 
anticholinergic,  Beta2-agonist/LA,  and  methyixanthine.  Short-acting  rescue  medication 
was  Beta2-agonist. 

ANOVA  analysis  (i.e.,  an  analysis  of  variance  between  groups)  con¬ 
firmed  significant  variation  among  the  ten  MTFs  for  each  of  the 
measures.  The  baseline  performance  information  for  the  MTFs  is 
reported  in  Chapter  Three,  including  figures  with  MTF  comparisons 
for  each  of  the  six  measures  and  testing  of  the  statistical  significance 
of  differences  among  them  in  performance  on  each  measure. 

Descriptions  of  Trends  for  Indicators.  To  describe  trend  infor¬ 
mation,  we  prepared  tables  and  figures  displaying  estimates  for  the 
indicators  over  the  two  study  years,  aggregated  separately  for  the 
demonstration  and  control  sites.  Recognizing  that  the  participating 
MTFs  focused  their  efforts  to  implement  the  guideline  in  one  or  two 
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clinics,  we  also  performed  separate  analyses  that  compared  perfor¬ 
mance  on  the  indicators  for  a  “target  group”  of  asthma  patients 
served  hy  those  clinics  with  performance  for  asthma  patients  served 
hy  other  clinics  in  the  MTFs. 

In  cases  where  we  found  substantial  differences  in  performance 
levels  or  trends  among  the  demonstration  or  control  sites,  we  exam¬ 
ined  trends  separately  for  each  demonstration  site  or  examined  aggre¬ 
gate  trends  for  the  relevant  group  of  sites  after  excluding  an  MTF 
with  outlying  values.  The  quantitative  results  were  compared  to  the 
implementation  strategies  of  the  demonstration  sites  to  better  inter¬ 
pret  the  observed  trends.  This  step  allowed  us  to  assess  the  extent  to 
which  those  strategies  were  reflected  in  observed  service  changes  (or 
did  not). 

Testing  the  Significance  of  Indicator  Trends.  The  final  step  of 
the  analysis  was  to  test  whether  observed  changes  in  service  rates  or 
medication  use,  if  any,  were  large  enough  to  be  statistically  signifi¬ 
cant,  after  controlling  for  temporal  trends  and  for  patient  characteris¬ 
tics.  For  each  indicator,  we  estimated  a  regression  model  with  the 
dependent  variable  being  the  indicator  of  interest  and  the  predictor 
variables,  including  a  dichotomous  variable  for  demonstration  or  con¬ 
trol,  a  set  of  dummy  variables  for  the  quarter-year  periods,  and  vari¬ 
ables  for  the  patient  characteristics. 

To  test  for  changes  in  the  indicator  for  the  demonstration  sites 
between  the  baseline  and  intervention  periods,  we  also  included  one 
or  more  interaction  terms  for  demonstration  sites  and  each  of  the 
three  quarters  of  the  intervention  period.  To  determine  the  final 
specification  of  the  interaction  terms,  we  were  guided  by  the  observed 
trends  for  the  measures  and  the  significance  of  the  coefficients  on  the 
interaction  term  for  each  quarter.  The  results  of  the  trend  analyses  are 
reported  in  Chapter  Five. 


Estimating  the  Costs  of  Care 

The  analysis  of  costs  of  care  had  two  purposes:  to  gain  an  under¬ 
standing  of  the  costs  that  MTFs  incur  for  health-care  services  for 
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asthma  patients  and  to  evaluate  whether  introduction  of  the  asthma 
practice  guideline  had  ohservahle  effects  on  those  costs  for  MTFs  par¬ 
ticipating  in  the  demonstration.  The  sample  used  for  the  analysis 
consisted  of  asthma  patients  served  hy  the  demonstration  MTFs  (the 
MTF  user  population).  The  analysis  distinguished  between  costs 
associated  with  the  MTF  user  and  enrollee  groups. 

To  estimate  the  costs  of  care,  we  used  MEPRS  financial  data  to 
develop  sets  of  unit  costs  for  different  types  of  inpatient  and  out¬ 
patient  encounters.  The  relevant  estimated  unit  cost  then  was  applied 
to  each  unit  of  service  in  the  SIDR  and  SADR  encounter  records 
(where  each  record  represented  one  unit  of  service). 

In  developing  our  unit-cost  estimation  methodology,  we  wanted 
to  derive  cost  estimates  that  captured  all  MTF  costs  of  care  for  inpa¬ 
tient  or  outpatient  events  and  were  sensitive  to  variations  in  the 
intensity  of  resources  required  to  provide  health  care  of  different 
types.  This  cost  estimation  method  was  developed  originally  as  part  of 
the  RAND  evaluation  of  the  Medicare-DoD  Subvention  Demonstra¬ 
tion  (Farley,  Harris,  et  ah,  2003). 

We  designed  the  cost  estimation  methodology  with  technical 
consultation  from  SRA  International,  the  TRICARE  Management 
Activity  contractor  that  developed  the  Patient-Eevel  Cost  Allocation 
(PECA)  method  used  to  design  the  financial  provisions  of  the  dem¬ 
onstration.  The  methodology  we  developed  is  an  adaptation  of  the 
approach  SRA  took  in  developing  the  PECA  method. 

Eor  this  cost  analysis,  we  used  cost  and  workload  data  for  fiscal 
year  (EY)  1998  that  SRA  generated  for  MTE  outpatient  clinics  or 
inpatient  wards  for  all  Army  MTEs  included  in  this  evaluation  of  the 
asthma  guideline  demonstration.  Refer  to  Appendix  B  for  details  on 
the  calculation  of  the  unit  costs  and  their  application  to  each  MTE 
inpatient  and  outpatient  encounter  record. 

We  note  that  some  criticism  has  arisen  within  the  DoD  that  the 
MEPRS  data  overestimates  the  MTEs’  costs  of  doing  business.  The 
source  of  this  criticism  is  a  reported  lack  of  documentation  of  vaca¬ 
tion  time  as  well  as  overestimation  of  the  available  hours  of  military 
personnel  time  for  patient  care  activities.  (Military  personnel  spend 
some  of  their  normal  work  time  on  military-related  activities,  which 
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they  should  record  separately  from  their  patient  care  activities.  How¬ 
ever,  many  do  record  it  as  patient  care  time,  thus  inflating  their 
reported  time  availahility.)  While  acknowledging  this  issue,  we  also 
understand  that  MEPRS  offers  the  best  available  data,  and  it  is  the 
basis  for  all  other  cost  estimations  for  the  demonstration. 

We  updated  the  unit  costs  to  FY  1999  estimates  by  applying  an 
inflation  factor  of  1.4  percent.  These  same  unit  costs  were  applied  to 
encounters  for  both  study  years.  By  holding  unit  costs  constant,  any 
observed  changes  in  costs  between  study  years  one  and  two  can  be 
attributed  to  changes  in  utilization.  We  tested  two  references  for  cost 
increases  to  determine  the  1.4  percent  inflation  rate,  which  are 
described  further  in  Appendix  B. 

Separate  unit  costs  were  applied  to  each  MTF  encounter  for  dif¬ 
ferent  types  of  hospital  inpatient  stays  (e.g.,  medical,  surgical)  or  out¬ 
patient  visits  to  different  types  of  clinics.  Then  we  aggregated  all 
encounters  for  each  asthma  patient  in  the  study  to  the  patient  level, 
which  we  used  to  analyze  per-capita  costs.  Costs  also  were  analyzed  at 
the  encounter  level  to  assess  the  distribution  of  MTF  costs  between 
TRICARE  Prime  enrollees  and  other  beneficiaries,  as  well  as  to  assess 
the  extent  to  which  costs  are  distributed  between  MTFs  and  network 
providers  in  the  community.  From  the  patient  perspective,  we  exam¬ 
ined  the  total,  inpatient,  and  outpatient  costs  of  care  per  patient  for 
MTF  services,  looking  separately  at  MTF  enrollees  and  nonenrollees. 

In  the  next  chapter,  we  describe  the  patient  population  and  pro¬ 
vide  baseline  performance  for  the  outcome  measures  for  effective 
asthma  care. 


CHAPTER  THREE 

Asthma  Populations  and  Practices  at  the  Baseline 


An  understanding  of  the  patient  population  and  baseline  performance 
for  key  care  measures  is  critical  for  designing  guideline  implementa¬ 
tion  strategies  by  the  demonstration  MTFs  and  evaluating  the  impact 
of  their  implementation  of  those  strategies. 

Asthma  patients  need  ongoing  care  management.  Therefore, 
TRICARE  beneficiaries  with  asthma  are  regular  users  of  services  cov¬ 
ered  by  their  TRICARE  benefits.  These  services  may  be  provided  by 
MTEs  or  network  providers,  depending  on  the  patients’  TRICARE 
Prime  enrollment  status  and  the  availability  of  needed  services  at  the 
MTEs.  As  discussed  earlier,  each  MTE  serves  asthma  patients  who  are 
enrolled  in  TRICARE  Prime  at  the  MTE  as  its  primary  patient 
population  as  well  as  other  patients  who  are  not  in  Prime  or  who  are 
enrolled  at  another  location  but  need  care  while  in  the  area  or  are 
referred  there  from  another  MTE  or  network  provider. 

In  this  chapter,  we  first  report  descriptive  information  on  the 
services  provided  to  the  total  population  of  asthma  patients  served  by 
Army  MTEs.  This  information  was  developed  to  better  understand 
the  mix  of  asthma  patients  served  by  the  Army  MTEs  and  to  charac¬ 
terize  how  they  use  MTEs  or  other  community-based  providers  for 
their  health-care  services. 

We  then  describe  the  baseline  performance  of  the  asthma 
patients  enrolled  in  TRICARE  Prime  at  the  MTEs  that  were  demon¬ 
stration  sites  or  control  sites  (MTE  enrollees).  This  baseline  data 
shows  how  much  variation  exists  in  the  asthma  indicators  across  the 
study  MTEs,  which  can  help  focus  interventions  to  achieve  greater 
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consistency  in  practices  across  the  system.  Baseline  performance 
information  provides  a  context  for  assessing  strategies  to  improve  per¬ 
formance  as  well  as  effects  of  using  the  practice  guideline,  which  are 
examined  in  the  next  two  chapters. 


The  Asthma  Population  Served  by  Army  MTFs 

Characteristics  of  the  Total  Asthma  Population 

An  estimated  121,465  asthma  patients  resided  in  the  service  areas  of 
Army  MTFs  and  were  served  hy  either  the  Army  MTFs  or  local  net¬ 
work  providers  during  study  year  one  (CY  1999)  and  a  slightly 
smaller  number  of  120,980  patients  were  served  during  study  year 
two  (CY  2000),  as  shown  in  Table  3.1.  This  table  shows  the  percent¬ 
ages  of  asthma  patients  identified  by  using  each  of  the  encounter  data 
sources,  in  the  order  by  which  counts  for  the  data  sources  are  dis¬ 
played  in  the  table.  For  study  year  one,  for  example,  we  were  able  to 
identify  52.7  percent  of  the  patients  using  the  SIDR  and  SADR  data 
and  we  identified  an  additional  27.0  percent  (for  whom  no  SIDR  or 
SADR  data  existed)  by  adding  the  PEC  pharmacy  data  to  the  process. 
The  network  provider  data  added  another  16.8  percent  of  patients, 
and  the  NMOP  data  added  the  remaining  3.5  percent.  Similar  per¬ 
centages  were  found  for  study  year  two. 

The  data  indicate  whether  asthma  patients  were  active-duty 
Army  personnel  or  other  service  personnel  or  military  retirees  or  fam¬ 
ily  members.  Among  those  affiliated  with  the  Army,  the  largest  frac¬ 
tion  (43,353  in  study  year  one  and  42,066  in  study  year  two)  was 
family  members  of  active-duty  personnel  and  the  next  largest  group 
was  family  members  of  retirees  (17,905  in  study  year  one  and  18,069 
in  study  year  two).  Active-duty  Army  personnel  were  the  smallest 
group  (15,619  in  study  year  one  and  15,796  in  study  year  two).  More 
than  32,000  of  the  asthma  patients  were  identifiable  as  personnel, 
retirees,  or  family  members  of  personnel  in  services  other  than  the 
Army.  For  the  subset  of  patients  only  identified  from  the  NMOP 
data,  information  on  their  military  status  was  not  available  in  the  data 
obtained  so  they  are  reported  separately  in  the  table. 
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Table  3.1 

Identification  of  the  Asthma  Population  Served  by  Army  MTFs  or  Network 
Providers  in  Army  MTF  Catchment  Areas,  by  Study  Year 


SIDR/SADR 

PEC 

Pharmacy 

Network 

Provider 

NMOP 

Total 

study  year  one 

Army  active-duty 

11,782 

3,733 

104 

a 

15,619 

Army  family  member 

30,632 

8,534 

4,187 

a 

43,353 

Army  retired 

2,561 

4,820 

663 

a 

8,044 

Retired  family  mem¬ 
ber 

7,407 

7,340 

3,158 

a 

17,905 

Other  services 

11,601 

8,411 

12,274 

a 

32,286 

NMOP  only 

a 

a 

a 

4,258 

4,258 

Total  patients,  year 
one 

63,983 

32,838 

20,386 

4,258 

121,465 

Percentage  of  total 

52.7 

27.0 

16.8 

3.5 

100.0 

Study  year  two 

Army  active-duty 

12,004 

3,682 

110 

a 

15,796 

Army  family  member 

30,380 

7,620 

4,066 

a 

42,066 

Army  retired 

2,549 

4,958 

713 

a 

8,220 

Retired  family  mem¬ 
ber 

7,189 

7,624 

3,256 

a 

18,069 

Other  services 

11,709 

8,344 

12,462 

a 

32,515 

NMOP  only 

a 

a 

a 

4,314 

4,314 

Total  patients,  year 
two 

63,831 

32,228 

20,607 

4,314 

120,980 

Percentage  of  total 

52.8 

26.6 

17.0 

3.6 

100.0 

“  NMOP  claims  lacked  data  on  the  military  status  of  beneficiaries,  so  we  could  not  clas¬ 
sify  any  patients  who  were  identified  only  through  NMOP  claims. 

NOTE:  Study  year  one  is  January  through  December  1999  and  study  year  two  is  Janu¬ 
ary  through  December  2000. 

The  demographic  characteristics  of  these  asthma  patients  are 
reported  in  Table  3.2.  Overall,  16.9  percent  of  the  patients  in  study 
year  one  were  less  than  6  years  of  age,  23.1  percent  were  6  to  17  years 
of  age,  30.8  were  18  to  44  years  of  age,  and  16.8  percent  were  45  to 
64  years  of  age.  The  elderly  represented  a  fraction  (12.3  percent)  of 
total  patients.  The  percentages  were  similar  for  study  year  two. 


Enrollment  Status  and  Use  of  MTF  Services 

The  enrollment  status  of  asthma  patients  seen  at  MTFs  was  also 
examined,  for  two  reasons.  First,  MTFs  should  manage  care  more 
effectively  for  their  enrollees  than  for  intermittent  users  who  may  he 
obtaining  care  from  local  network  providers  or  other  MTFs.  There- 
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Table  3.2 

Demographic  Characteristics  of  the  Asthma  Population  Served  by  Army 
MTFs  or  Network  Providers  in  Army  MTF  Catchment  Areas,  by  Study  Year 


Army 

Army 

Retired 

All 

Active- 

Family 

Army 

Family 

Other 

NMOP 

Patients 

Duty 

Member 

Retired 

Member 

Services 

Only 

study  year  one 
Distribution  by  age 
Total 

121,457 

15,619 

43,347 

8,044 

17,905 

32,284 

4,258 

Less  than  6  years 

20,531 

4 

14,567 

0 

604 

5,343 

13 

6-17 

28,096 

48 

15,956 

1 

3,910 

8,001 

180 

18-44  years 

37,465 

14,695 

11,391 

883 

2,939 

7,305 

252 

45-64  years 

20,390 

850 

1,159 

3,550 

6,291 

7,077 

1,463 

65+  years 

Distribution  by  gen¬ 

14,975 

22 

274 

3,610 

4,161 

4,558 

2,350 

der 

Total 

121,452 

15,618 

43,352 

8,039 

17,904 

32,281 

4,258 

Male  (%) 

47.6 

71.7 

44.3 

94.6 

18.5 

45.3 

44.2 

Study  year  two 
Distribution  by  age 
Total 

119,980 

15,796 

42,066 

7,220 

18,069 

32,515 

4,314 

Less  than  6  years 

18,679 

3 

13,267 

0 

524 

4,879 

6 

6-17  years 

28,272 

55 

16,075 

0 

3,766 

8,209 

167 

18-44  years 

37,437 

14,822 

11,239 

878 

2,934 

7,306 

258 

45-64  years 

20,768 

891 

1,203 

3,592 

6,400 

7,300 

1,382 

65+  years 

Distribution  by  gen¬ 

14,824 

25 

282 

2,750 

4,445 

4,821 

2,501 

der 

Total 

120,973 

15,795 

42,066 

8,217 

18,068 

32,513 

4,314 

Male  (%) 

47.0 

70.6 

43.9 

94.5 

17.6 

44.7 

42.4 

NOTE:  Study  year  one  is  January  through  December  1999  and  study  year  two  is  Janu¬ 
ary  through  December  2000.  The  finding  of  active-duty  and  retired  personnel  under 
18  may  represent  errors  in  the  records. 


fore,  MTFs  should  be  held  accountable  first  for  care  to  their  own 
enrollees,  and  any  performance-monitoring  system  should  use  mea¬ 
sures  calculated  specifically  for  this  population.  Second,  for  accurate 
calculation  of  performance  measures,  all  care  obtained  by  each  asthma 
patient  group  should  be  taken  into  account.  For  example,  as  MTF- 
based  PCMs  manage  asthma  care  for  their  enrollees,  they  may  refer 
them  to  specialty  care  by  physicians  at  other  MTFs  or  by  other  pro¬ 
viders.  Further,  a  patient  may  have  an  exacerbation  while  away  from 
home  and  use  an  out-of-town  emergency  room  for  care.  These  events 
all  need  to  be  included  in  the  measures  for  use  rates  for  those  services. 
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Our  analysis  revealed  the  importance  of  using  both  MTF  service 
encounter  data  and  network  provider  claims  to  gain  a  full  under¬ 
standing  of  service-use  patterns.  As  shown  for  study  year  one  in  Table 
3.3,  80.2  percent  of  MTF  outpatient  or  emergency  room  visits  were 
provided  for  members  of  the  asthma  patient  population  enrolled  in 
TRICARE  Prime  at  the  MTFs  (the  numbers  for  study  year  two  are 
similar).  Another  19.1  percent  of  the  visits  were  for  nonenrolled 
patients,  and  less  than  1  percent  of  services  were  for  patients  enrolled 
with  network  providers.  Thus,  a  substantial  share  of  MTF  health-care 
resources  were  dedicated  to  providing  outpatient  services  for  nonen- 


Table  3.3 

Enrollment  Status  for  Patients  Receiving  Asthma  Care  at  Army  MTFs  or 
Network  Providers  in  Army  MTF  Catchment  Areas,  by  Study  Year 


Asthma  Outpatient/ 

ER  Visits  Asthma  Admissions 


Number  of  Percentage  Number  of  Percentage 
Encounters  of  Total  Encounters  of  Total 


Study  year  one 
MTF  service 


MTF  enrollee 

103,308 

80.2 

4,971 

98.9 

Network  enrollee 

908 

0.7 

51 

1.0 

Not  enrolled 

24,666 

19.1 

2 

0.0 

All  patients 

128,882 

100.0 

5,024 

100.0 

Network  provider  service 

MTF  enrollee 

24,004 

37.1 

326 

12.6 

Network  enrollee 

24,745 

38.3 

1,186 

45.9 

Not  enrolled 

15,892 

24.6 

1,072 

41.5 

All  patients 

64,641 

100.0 

2,584 

100.0 

Study  year  two 

MTF  service 

MTF  enrollee 

110,661 

83.8 

4,242 

98.5 

Network  enrollee 

831 

0.6 

60 

1.4 

Not  enrolled 

20,530 

15.6 

3 

0.1 

All  patients 

132,022 

100.0 

4,305 

100.0 

Network  provider  service 

MTF  enrollee 

23,334 

34.3 

273 

11.9 

Network  enrollee 

31,310 

46.1 

1,039 

45.2 

Not  enrolled 

13,299 

19.6 

985 

42.9 

All  patients 

67,943 

100.0 

2,297 

100.0 

NOTE:  MTF  enrollee  refers  to  individuals  enrolled  in  TRICARE  Prime  with  a  primary 
care  manager  based  at  an  MTF.  Network  enrollee  refers  to  individuals  enrolled  in 
TRICARE  Prime  with  a  PCM  who  is  a  network  provider. 
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rollees.  By  contrast,  an  MTF’s  own  enrollees  accounted  for  virtually 
all  inpatient  care  provided  by  MTFs. 

The  patterns  differed  for  network  provider  services.  An  esti¬ 
mated  38.3  percent  of  network  provider  outpatient  and  emergency 
room  visits  were  provided  to  patients  enrolled  in  TRICARE  Prime 
with  network  providers.  Another  37.1  percent  were  provided  for 
patients  enrolled  at  MTFs,  and  24.6  percent  were  provided  to  non- 
enrollees.  A  different  distribution  was  found  for  network  provider 
inpatient  services,  with  45.9  percent  provided  to  network  provider 
enrollees,  12.6  percent  provided  to  MTF  enrollees,  and  another  41.5 
percent  provided  to  nonenrollees.  These  results  are  displayed  graphi- 


Figure  3.1 

Enrollment  Status  of  Asthma  Patients  for  Outpatient  and  Emergency 
Room  Visits  to  MTF  and  Network  Providers,  by  Study  Year 
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Figure  3.2 

Enrollment  Status  of  Asthma  Patients  for  Inpatient  Admissions  at  MTFs 
and  Network  Providers,  by  Study  Year 
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cally  for  outpatient  and  emergency  room  visits  in  Figure  3.1  and  for 
inpatient  stays  in  Figure  3.2. 

Figures  3.3  and  3.4  provide  a  view,  respectively,  into  outpatient 
and  emergency  room  and  inpatient  admission  use  patterns  from  the 
patient’s  perspective.  The  two  figures  show  the  distribution  of 
encounters  hy  provider  type  for  each  type  of  enrollment  status.  The 
data  illustrate  that  patients  enrolled  at  MTFs  use  a  mix  of  MTF  and 
network  provider  services,  confirming  that  all  of  these  services  must 
he  considered  to  obtain  accurate  estimates  of  their  health-care  use 


rates. 
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Baseline  Performance  on  Asthma  Care  Measures  at  the 
Study  MTFs 

The  number  of  asthma  patients  who  used  each  of  the  demonstration 
and  control  sites  is  reported  in  Table  3.4.  These  data  include  separate 
counts  of  all  patients  who  used  each  MTF  at  least  once  during  each 
study  year  (MTF  users)  and  the  subset  of  patients  who  were  enrolled 
at  the  MTF  all  year  (MTF  enrollees),  as  indicated  by  enrollment 
status  codes  on  their  encounter  records.  The  overall  percentages  of 
MTF  users  who  were  MTF  enrollees  were  similar  for  the  demonstra¬ 
tion  and  control  sites.  For  example,  during  study  year  one,  4,631  out 
of  the  8,439  MTF  users  for  the  demonstration  MTFs  (54.9  percent) 


Figure  3.3 

Outpatient  and  Emergency  Room  Visits  to  MTFs  and  Network  Providers, 
by  Enrollment  Status  of  Asthma  Patients  and  Study  Year 
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Figure  3.4 

Inpatient  Admissions  at  MTFs  and  Network  Providers,  by  Enrollment 
Status  of  Asthma  Patients  and  Study  Year 
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were  enrolled  in  TRICARE  Prime  at  the  sites.  By  comparison,  8,339 
out  of  the  14,640  MTF  users  at  the  control  MTFs  (57.0  percent) 
were  enrolled  there.  The  total  size  of  the  patient  sample  for  the  con¬ 
trol  MTFs  was  larger  than  the  sample  for  the  demonstration  MTFs 
because  there  were  six  control  MTFs  and  only  four  demonstration 
MTFs.  However,  the  average  numbers  of  patients  per  MTF  for  the 
two  groups  were  similar  for  both  MTF  users  and  enrollees. 

Using  the  MTF  enrollee  group  of  patients  (those  enrolled  in 
TRICARE  Prime  at  a  study  MTF),  which  was  the  study  sample  for 
the  effects  analysis,  we  calculated  average  values  of  the  six  guideline 
effect  indicators  for  the  baseline  time  period  of  January  through 
December  1999  (study  year  one).  We  calculated  separate  values  for 
each  of  the  10  MTFs  in  the  study  as  well  as  an  overall  average  for 
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Table  3.4 

Asthma  Population  Using  the  Demonstration  and  Control  MTFs, 
by  Study  Year 


Study  Year  One 

Study  Year  Two 

study  Site 

MTF 

Users 

MTF 

Enrollees 

MTF 

Users 

MTF 

Enrollees 

Demonstration  sites 

Demo  1 

2,295 

1,176 

2,375 

1,059 

Demo  2 

2,590 

1,159 

2,469 

1,043 

Demo  3 

2,406 

1,715 

2,689 

1,628 

Demo  4 

1,148 

581 

1,524 

559 

All  demos 

8,439 

4,631 

9,057 

4,289 

Patients  per  site 

2,110 

1,158 

2,264 

1,072 

Control  sites 

Control  1 

2,380 

1,635 

2,097 

1,280 

Control  2 

1,729 

785 

1,740 

659 

Control  3 

1,401 

736 

1,394 

595 

Control  4 

4,631 

2,621 

3,927 

2,050 

Control  5 

2,853 

1,747 

2,676 

1,558 

Control  6 

1,646 

815 

2,489 

889 

All  controls 

14,640 

8,339 

14,323 

7,031 

Patients  per  site 

2,440 

1,390 

2,387 

1,172 

NOTE:  MTF  enrollee  signifies  beneficiaries  who  were  enrolled  in  TRICARE 
Prime  at  this  MTF  for  all  of  their  inpatient  and  outpatient  encounters  in 
the  SIDR  and  SADR  data. 


each  indicator,  which  was  a  henchmark  against  which  the  value  for 
each  MTF  could  he  compared. 

Table  3.5  lists  the  six  outcome  measures  and  describes  the 
rationale  provided  for  each  by  the  asthma  guideline.  Wide  variation 
across  MTFs  on  any  given  measure  suggests  that  MTFs  may  not  be 
providing  care  consistently,  which  could  include  overtreatment  in 
some  cases  and  undertreatment  in  others. 

Figures  3.5  through  3.10  show  baseline  performance  of  the 
study  MTFs  on  the  six  indicators  of  asthma  care.  The  first  bar  on  the 
left  of  each  figure  indicates  the  overall  average  baseline  performance 
for  all  ten  MTFs,  and  the  remaining  bars  show  the  values  for  individ¬ 
ual  MTFs.  To  protect  the  confidentiality  of  the  MTFs,  the  results  are 
reported  anonymously. 

We  tested  the  statistical  significance  of  the  differences  of  MTF 
values  by  comparing  each  MTF’s  average  value  for  a  measure  to  the 
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Table  3.5 

Rationale  for  Asthma  Indicators  Provided  by  the  Guideline 


Indicator 

Guideline  Direction 

Long-term  controllers 

For  asthma  more  severe  than  mild  intermittent,  long¬ 
term  controller  medications  are  to  be  used  according 
to  severity  level.  Very  low  percentages  of  patients  on 
controllers  may  signal  inadequate  preventive  care. 

Complementary  medica- 

Complementary  medications  support  long-term  control 

tions 

of  asthma  for  some  patients. 

Short-acting  rescue 

Patients  should  have  prescriptions  for  rescue  medica¬ 
tions  for  response  to  exacerbations. 

Outpatient  visits 

Asthma  is  to  be  managed  proactively  according  to 
severity  level.  Very  low  visit  rates  may  signal  inade¬ 
quate  care  management. 

Emergency  room  visits 

Proactive  management  of  asthma  should  reduce  emer¬ 
gency  visit  rates  for  exacerbations.  High  visit  rates 
indicate  a  need  for  more  prevention. 

Figure  3.5 

Baseline  Percentage  of  Asthma  Patients  Prescribed  Long-Term  Controller 
Medications,  Total  and  by  Site 
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*p  =  less  than  0.05,  **p  =  less  than  0.0. 
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Figure  3.6 

Baseline  Percentage  of  Asthma  Patients  Prescribed  Complementary 
Medications,  Total  and  by  Site 
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*p  =  less  than  0.05,  **p  =  less  than  0.01. 
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average  value  for  the  remaining  nine  MTFs.  When  the  performance 
of  an  MTF  differs  significantly  from  the  average  of  the  other  MTFs, 
the  MTF’s  label  in  the  legend  is  followed  hy  asterisks  (*  for  p  <  0.05, 
**  for  p  <  0.01).  As  discussed  in  Chapter  Two,  both  the  clinical  sig¬ 
nificance  of  observed  differences  among  MTFs  and  the  statistical  sig¬ 
nificance  of  these  differences  should  be  considered  when  interpreting 
these  results. 

Long-Term  Controllers.  We  find  moderate  variation  across  the 
study  MTFs  in  the  baseline  percentages  of  asthma  patients  who  were 
treated  with  other  long-term  controller  medications  to  reduce  the  fre¬ 
quency  of  asthma  exacerbations.  As  shown  in  Figure  3.5,  the  percent¬ 
ages  of  patients  on  these  medications  averaged  43  percent  and  ranged 
between  35  and  55  percent  across  the  study  sites.  Not  all  of  the  dif¬ 
ferences  observed  were  statistically  significant. 
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Complementary  Medications.  The  percentages  of  asthma 
patients  prescrihed  complementary  medications  were  lower  than  the 
percentages  with  long-term  controllers,  and  these  percentages  varied 
more  widely  across  the  study  sites,  as  shown  in  Figure  3.6.  For  all 
sites,  an  overall  average  of  21  percent  of  patients  had  been  prescrihed 
complementary  medications,  ranging  from  14  percent  to  27  percent 
across  the  individual  sites.  The  size  of  these  differences  were  statisti¬ 
cally  significant  for  seven  of  the  ten  MTFs. 

Short-Acting  Rescue  Medications.  Asthma  patients  were  more 
likely  to  have  prescriptions  for  short-acting  rescue  medications  than 
the  longer-term  controller  medications.  Across  all  MTFs,  Figure  3.7 
shows  that  an  overall  average  of  65  percent  of  patients  had  been  pre- 


Figure  3.7 

Baseline  Percentage  of  Asthma  Patients  Prescribed  Short-Acting  Rescue 
Medications,  Total  and  by  Site 
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scribed  a  rescue  medication,  with  values  at  individual  MTFs  ranging 
from  55  percent  to  79  percent  of  patients.  This  range  represents 
moderate  variation  across  facilities,  similar  to  that  for  the  long-term 
controllers.  This  also  is  reflected  in  the  smaller  number  of  MTFs  for 
which  the  size  of  differences  were  statistically  significant. 

Outpatient  Visits.  For  asthma-related  outpatient  visits,  an  overall 
average  of  165  visits  per  100  patients  occurred,  as  shown  in  Figure 
3.8.  Rates  for  the  individual  MTFs  ranged  from  135  visits  to  200 
visits  per  1 00  patients,  and  about  half  of  the  differences  among  MTFs 
are  statistically  significant. 

Emergency  Room  Visits  and  Hospital  Inpatient  Stays.  We  find 
variation  across  MTFs  for  both  measures,  which  is  statistically  signifi¬ 
cant  for  many  of  the  MTFs,  as  shown  for  emergency  room  visits  in 


Figure  3.8 

Baseline  Annual  Asthma-Related  Outpatient  Visits  per  100  Asthma  Patients, 
Total  and  by  Site 
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Figure  3.9  and  for  hospital  inpatient  stays  in  Figure  3.10.  Facilities 
with  rates  on  these  measures  significantly  higher  than  those  of  other 
MTFs  are  candidates  for  assessment  to  analyze  the  contributing  fac¬ 
tors  to  the  high  rates  and  identify  actions  that  might  he  taken  to 
improve  prevention  of  asthma  exacerbations  and  related  incidence  of 
emergency  room  use  or  hospitalizations. 


Summary 

The  baseline  data  analysis  produced  the  following  findings. 


•  The  asthma  population  served  by  Army  MTFs  is  large  and  con¬ 
sists  primarily  of  Army  family  members,  individuals  affiliated 


Figure  3.9 

Baseline  Annual  Asthma-Related  Emergency  Room  Visits  per  100  Asthma 
Patients,  Total  and  by  Site 
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Inpatient  stays  per  100  patients 
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Figure  3.10 

Baseline  Annual  Asthma-Related  Hospitalizations  per  100  Asthma  Patients, 
Total  and  by  Site 
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with  other  military  services,  and  family  members  of  retirees. 
Patients  are  fairly  evenly  distributed  across  age  groups.  The 
asthma  populations  served  by  the  individual  MTFs  vary  widely 
in  size,  depending  on  the  sizes  and  characteristics  of  the  benefi¬ 
ciary  popularions  residing  in  their  catchment  areas. 

•  Although  the  majority  of  asthma  patient  outpatient  or  emer¬ 
gency  room  visits  involved  TRICARE  Prime  beneficiaries  who 
were  enrolled  at  the  MTF  that  provided  care,  a  substantial  por¬ 
tion  of  patients  seen  were  enrolled  at  other  facilities  (MTFs  or 
civilian  network  sites).  By  contrast,  each  MTF’s  own  enrollees 
accounted  for  virtually  all  inpatient  care  provided  that  MTF. 

•  The  baseline  comparisons  of  outcome  measures  for  the  study 
sites  show  that  outcome  measures  vary  only  moderately  across 
the  MTFs  for  many  of  the  indicators  at  baseline.  For  the  three 
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medication  indicators,  in  particular,  MTFs  had  similar  percent¬ 
ages  of  patients  using  each  type  of  medication. 

For  some  service-use  indicators,  such  as  emergency  room  care 
and  hospitalization,  one  or  two  MTFs  had  either  much  higher 
or  much  lower  levels  than  the  other  MTFs  in  the  sample.  How¬ 
ever,  the  importance  of  these  differences  depends  on  how  the 
actual  performance  at  each  site  varies  from  recommended 
guidelines,  where  applicable. 

The  baseline  measures  can  be  used  as  benchmarks  to  detect  dif¬ 
ferences  across  facilities  and  to  identify  areas  where  practice 
improvements  may  be  needed. 


CHAPTER  FOUR 

The  Guideline  Implementation  Process 


The  asthma  guideline  demonstration  was  the  second  of  the  three 
demonstrations  that  tested  an  implementation  approach  that  coordi¬ 
nated  actions  at  the  command  (MEDCOM)  and  local  (MTF)  levels 
to  achieve  best  clinical  practices.  MEDCOM  defined  the  desired 
clinical  practices  (as  specified  in  the  DoD/VA  practice  guideline)  and 
key  metrics  to  measure  attainment  of  those  practices;  provided  tools 
to  assist  the  MTFs  as  they  introduced  new  practices  in  response  to  the 
guideline;  and  facilitated  MTF  guideline  implementation  via  site  vis¬ 
its,  e-mail,  and  phone  communications. 

The  practice  changes  were  carried  out  hy  the  MTFs  as  the 
health-care  delivery  organizations.  The  MTFs  were  offered  the  flexi¬ 
bility  to  define  strategies  and  clinical  process  changes  within  the  con¬ 
text  of  their  respective  missions,  populations,  and  administrative  and 
clinical  assets.  Because  these  characteristics  differed  across  facilities, 
we  expected  to  observe  differences  among  the  MTFs’  implementation 
strategies  and  the  pace  at  which  they  introduced  practice  changes. 

This  chapter  reports  the  findings  of  the  process  evaluation  in 
terms  of  the  infrastructure  established  for  the  asthma  guideline  dem¬ 
onstration  and  the  strategies  and  actions  undertaken  by  the  MTFs  to 
implement  best  practices  for  management  of  asthma  patients.  We 
first  describe  the  MEDCOM  support  structure  and  activities.  Next, 
we  describe  the  MTF  environment  and  support  structure  for  guide¬ 
line  implementation  at  the  participating  MTFs,  including  support  by 
the  MTF  command  team  and  roles  of  the  guideline  champions, 
facilitators,  and  implementation  teams.  Then  we  describe  the  strate- 
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gies  and  actions  of  the  MTF  implementation  teams,  as  identified  in 
their  action  plans,  and  we  assess  the  progress  they  made  in  achieving 
desired  practice  changes.  A  summary  of  key  findings  in  the  process 
evaluation  is  included  at  the  end  of  the  chapter. 


MEDCOM  Support 

The  system-level  responsihility  for  operating  the  AMEDD  program 
for  evidence-hased  practice  guidelines  was  assigned  to  the  MEDCOM 
Quality  Management  Directorate.  Initially,  the  staff  for  this  new  ini¬ 
tiative  consisted  of  a  full-time  program  director  and  a  secretary.  By 
the  time  the  asthma  guideline  demonstration  began,  the  MEDCOM 
program  staff  had  been  expanded,  with  the  addition  of  three  full-time 
guideline  representatives  to  support  MTEs  in  implementing  the  three 
demonstrations  as  well  as  other  practice  guidelines  that  were  being 
introduced  for  implementation  across  the  AMEDD  system. 

The  asthma  practice  guideline  demonstration  had  the  advantage 
of  building  on  the  lessons  learned  from  the  low  back  pain  guideline 
demonstrations,  which  were  shared  with  the  participating  MTEs 
during  the  kickoff  conference  (described  below)  and  throughout  the 
MTEs’  implementation  activities  during  the  demonstration. 

MEDCOM  supported  the  MTEs  in  implementing  the  asthma 
practice  guideline  by  organizing  an  offsite  kickoff  conference  to 
introduce  the  implementation  teams  to  the  guideline  and  help  them 
develop  implementation  action  plans,  providing  the  MTEs  with  a 
toolkit  of  items  to  support  guideline  implementation,  and  encourag¬ 
ing  communications  and  technical  support  among  the  demonstration 
sites  and  MEDCOM.  We  briefly  describe  each  of  these  activities  and 
provide  our  findings  regarding  each  of  them. 

The  Kickoff  Conference 

The  MTE  implementation  teams  gathered  for  two  days  on  August 
19-20,  1999,  in  Augusta,  Georgia.  This  planning  conference  was  run 
by  MEDCOM  with  RAND  administrative  support.  A  total  of  46 
individuals  from  the  four  demonstration  MTEs  participated  in  the 
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conference,  and  five  individuals  from  other  MTFs  attended  as  observ¬ 
ers. 

The  meeting  began  with  a  half-day  of  plenary  sessions  at  which 
the  asthma  guideline  was  introduced  and  instructions  for  action  plan 
development  were  provided.  The  teams  used  the  majority  of  the 
meeting  to  develop  their  respective  implementation  action  plans. 
Each  MTF  team  had  designated  a  facilitator  who  guided  the  team 
through  a  four-step  planning  process  developed  by  RAND.  All  plans 
included  activities  to  introduce  the  guideline  and  implementation 
tools  to  both  providers  and  ancillary  staff,  plans  to  increase  patient 
education  and  case  management,  and  changes  needed  in  clinic  proce¬ 
dures  to  bring  asthma  care  into  compliance  with  the  guideline.  The 
MTFs  also  identified  the  metrics  they  planned  to  monitor  on  a  regu¬ 
lar  basis.  The  MTF  teams  briefed  the  representative  of  Southeast 
Regional  Command  on  their  action  plans  at  the  end  of  the  confer¬ 
ence. 

Participants  were  asked  to  complete  an  evaluation  form  at  the 
conference  to  provide  feedback  on  the  usefulness  of  the  conference 
and  to  make  suggestions  for  improvements.  We  obtained  an 
approximately  80  percent  response  rate  from  the  participating  MTF 
team  members.  In  general,  participants  reported  that  the  planning 
activities  at  the  kickoff  conference  were  useful  to  them.  They  also 
rated  very  highly  the  information  they  received  at  the  conference. 
Features  of  the  conference  that  participants  liked  best  included  the 
guideline  itself,  the  conference  notebook,  the  breakout  sessions,  the 
examples  of  implementation  plans  provided,  and  the  toolkit.  Partici¬ 
pants  suggested  that  future  kickoff  conferences  could  be  improved  by 
providing  the  clinical  practice  guideline  and  notebook  before  the  con¬ 
ference  and  by  allowing  more  time  for  the  breakout  sessions.  Some 
participants  suggested  that  the  conference  be  lengthened  by  up  to  one 
full  day  to  allow  more  time  for  planning  activities. 

The  Asthma  Toolkit 

In  preparation  for  the  asthma  guideline  demonstration,  MEDCOM 
and  the  Army  Center  for  Health  Promotion  and  Preventive  Medicine 
convened  a  toolkit  workshop  in  January  1999  with  the  participation 


56  Implementation  of  the  Asthma  Practice  Guideline  in  AMEDD 


of  about  40  providers  from  Army  MTFs  who  would  use  the  materials 
in  the  toolkits.  At  the  workshop,  the  providers  identified  and 
developed  tools  to  assist  Army  MTFs  in  working  with  the  asthma 
practice  guideline.  Using  the  products  generated  by  the  workshop 
participants,  MEDCOM  and  CHPPM  then  prepared  the  core  set  of 
implementation  tools. 

Table  4.1  provides  a  list  of  the  items  included  in  the  toolkit. 
These  tools  were  similar  to  those  provided  to  participants  in  the  low 
back  pain  demonstration,  although  the  asthma  tools  had  been  revised 
based  on  feedback  received  in  the  earlier  demonstration. 

Feedback  on  the  Forms.  Feedback  from  MTF  staff  on  the 
toolkit  items  was  obtained  during  group  interviews  and  focus  groups 
conducted  during  our  first  site  visits.  We  received  fewer  comments  on 
the  asthma  tools  than  we  had  on  similar  tools  used  in  the  low  back 
pain  demonstration,  which  may  reflect  improvements  to  the  tool 
development  process  that  had  been  made  in  response  to  feedback 
from  the  low  back  pain  demonstration.  However,  it  also  appeared 
that  the  MTF  staff  were  overwhelmed  by  the  large  number  of  items 
in  the  asthma  toolkit.  For  example,  few  of  the  providers  interviewed 
had  even  seen  all  of  the  tools.  This  lack  of  exposure  to  the  tools 
undoubtedly  contributed  to  the  low  use  of  these  items  made  by  the 
sites,  as  described  below. 

Encounter  Documentation  Form  701-R.  This  documentation 
form  included  a  section  on  vital  signs  to  be  filled  out  by  the  clinic 
staff,  a  self-assessment  and  current  medication  section  to  be  com¬ 
pleted  by  the  patient,  and  a  section  to  be  completed  by  the  health- 


Table  4.1 

Contents  of  the  Asthma  Toolkit 

Provider  Support 

Patient  Seif-Management  Support 

Encounter  documentation  form  701-R 
Master  problem  form  702-R 

Patient  action  plan  703-R 

Patient  education  form  704-R 

Sample  peak  flow  meter 

Reminder  cards  on  guideline  elements 

Pamphlets  on  asthma  self-management 
Videos  and  CD  on  asthma  education 

Three  videos  on  asthma  treatment 

CD  on  asthma  treatment 
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care  provider.  Primary  care  providers  participating  in  the  focus 
groups  reported  that  they  generally  liked  the  form,  although  at  our 
first  visit  they  suggested  that  more  blank  space  he  added  for  clinical 
notes.  They  also  suggested  that  the  patient  portion  of  the  form  he 
expanded  to  add  the  following  items:  coughing  as  a  symptom,  aller¬ 
gies,  hospitalization  history,  and  date  of  last  visit  to  ER. 

Despite  generally  positive  responses  to  the  Form  701-R  design, 
use  of  the  form  was  actively  encouraged  hy  only  one  MTF  implemen¬ 
tation  team,  and  this  site  was  the  only  one  to  actually  use  the  form 
consistently.  One  other  site  preferred  to  continue  using  another  form 
that  it  had  developed  prior  to  the  demonstration. 

At  the  other  two  sites,  use  of  Form  701-R  was  left  to  the  discre¬ 
tion  of  each  physician.  At  both  sites,  physicians  also  preferred  to  con¬ 
tinue  using  Form  600  (the  standard  cover  form  generated  for  each 
visit  and  attached  to  the  patient’s  medical  chart).  They  expressed 
preference  for  Form  600  because  it  provided  more  space  and  flexibil¬ 
ity  to  write  text  notes.  They  also  thought  Form  701-R  was  poorly 
suited  for  patients  with  multiple  diagnoses,  who  constituted  a  sig¬ 
nificant  share  of  their  patients.  At  one  of  these  sites,  ancillary  staff 
reported  that  use  of  Form  701-R  increased  the  time  required  to  pro¬ 
cess  patients  for  visits. 

Master  Problem  Form  702-R.  This  form  was  developed  to  record 
the  patient’s  history  of  tests,  emergency  room  visits,  inpatient  admis¬ 
sions,  triggers,  severity,  and  therapeutic  interventions.  Most  physi¬ 
cians  reported  they  never  had  seen  this  form,  and  none  of  the  clinics 
in  the  four  MTFs  used  it. 

Action  Plan  Form  703-R.  This  form  was  designed  as  a  handy  way 
to  comply  with  the  asthma  guideline  by  providing  a  written  treat¬ 
ment  plan  and  set  of  steps  for  the  patient  to  follow  in  the  event  the 
status  of  their  asthma  was  in  the  “green,”  “yellow,”  or  “red”  zone. 
The  form  was  to  be  signed  by  the  patient.  At  our  first  site  visit,  physi¬ 
cians  suggested  that  the  form  be  produced  with  a  carbon  copy  so  the 
patient  would  have  a  copy  and  another  copy  would  be  placed  in  the 
medical  chart.  Physicians  at  the  sites  rarely  used  this  form,  even 
though  those  who  had  seen  the  form  rated  it  highly.  Two  sites  con¬ 
tinued  to  use  action  plan  forms  developed  locally.  At  the  other  two 
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sites,  physicians  continued  to  prefer  to  devise  their  own  action  plans 
for  patients. 

Education  Form  704-R.  This  form  documented  the  patient’s 
asthma  status  and  knowledge  of  the  condition.  It  included  a  section 
to  he  filled  out  hy  the  patient  prior  to  the  education  class  and  a  sec¬ 
tion  to  he  filled  hy  the  instructor  at  the  end  of  the  class.  Two  sites 
liked  the  form  and  were  using  it.  The  other  two  sites  reported  they 
did  not  use  the  form,  although  one  of  them  was  using  the  SF  600  to 
document  that  a  patient  had  taken  an  asthma  education  class.  One 
site  suggested  that  the  form  he  modified  to  include  more  blank  space 
to  write  in.  It  also  reported  that  patients  had  difficulties  filling  out 
their  portion  of  the  form. 

Patient  Education  Video.  This  video  educates  patients  on  what 
asthma  is  and  how  they  can  manage  it  to  reduce  exacerbations.  The 
implementation  team  at  one  MTF  liked  the  patient  video  and 
showed  it  during  their  education  classes.  Two  other  MTF  teams  pre¬ 
ferred  to  continue  showing  videos  that  they  had  used  prior  to  the 
demonstration.  The  last  MTF  team  did  not  use  the  video  because  it 
thought  it  was  too  long  and  lacked  some  needed  information  about 
self-management  of  asthma. 

Patient  Education  Pamphlet.  This  pamphlet  provides  asthma 
patients  with  highlights  of  the  key  actions  they  can  take  to  help  man¬ 
age  their  asthma  effectively.  Participating  clinics  at  three  of  the  MTFs 
distributed  the  education  pamphlet  to  their  asthma  patients.  The 
fourth  MTF  did  not  distribute  it  because  they  felt  it  was  written  at 
too  high  a  reading  level  for  its  patient  population.  Implementation 
teams  and  other  clinic  staff  at  all  sites  recommended  that  patient  edu¬ 
cation  materials  be  written  at  the  fifth-grade  reading  level  and  make 
heavy  use  of  graphics  and  lighter  use  of  words.  It  also  was  recom¬ 
mended  that  patient  education  material  be  made  available  in  multiple 
languages,  including  German,  Korean,  and  Japanese. 

Guideline  Key  Elements  Cards.  These  cards  are  reminders  for 
providers  that  present  the  key  elements  and  algorithms  for  asthma 
management,  as  defined  by  the  asthma  practice  guideline.  Both  ver¬ 
sions  of  the  cards — the  desktop  cards  and  packet  laminated 
cards — were  rated  highly  by  the  physicians  who  had  seen  them.  At 
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our  first  site  visit,  it  was  suggested  the  actual  dosage  for  medications 
specific  to  each  level  of  severity  he  added  to  both  cards.  A  number  of 
providers  reported  at  the  first  site  visits  that  they  had  not  received 
these  cards.  When  we  followed  up  on  the  issue  at  our  second  visits, 
some  physicians  continued  to  report  they  had  not  received  these 
cards. 

Additional  Tool  Kit  Items.  At  the  first  round  of  visits,  some  MTF 
staff  indicated  they  would  like  to  have  a  PowerPoint  presentation 
available  designed  to  introduce  ancillary  staff  to  the  asthma  guideline 
and  the  staff  s  role  in  its  implementation.  Staff  also  suggested  that 
MEDCOM  provide  a  poster  displaying  the  asthma  guideline  key 
elements. 

Information  Exchange 

MEDCOM  created  two  information  exchange  mechanisms — a  list- 
serv  and  communication  support — to  help  MTE s  share  implementa¬ 
tion  experiences  and  learn  from  each  other. 

Listserv.  The  asthma  listserv  was  set  up  and  managed  by 
MEDCOM  staff  to  provide  communication  among  MEDCOM  and 
the  MTEs  participating  in  the  demonstration  about  new  develop¬ 
ments  in  asthma  care,  measures  and  data  for  monitoring  asthma  care 
quality,  and  other  MEDCOM  or  MTE  activities.  The  listserv  became 
operational  after  our  first  visit  to  the  sites,  and  any  staff  at  the  demon¬ 
stration  MTEs  could  register  to  participate  in  the  listserv,  at  their  dis¬ 
cretion. 

Staff  on  the  implementation  team  at  three  of  the  sites  generally 
knew  about  the  availability  of  the  listserv,  and  staff  at  the  fourth  site 
did  not  know  about  it.  Of  those  who  were  aware  of  the  listserv,  most 
had  either  never  accessed  it  or  had  accessed  it  only  once  or  twice. 
When  asked  why  not,  many  reported  they  were  too  busy  to  take  the 
time  to  use  it  or  were  not  interested.  However,  a  few  people  reported 
accessing  the  asthma  listserv  daily,  including  the  case  manager  at  one 
site  and  the  facilitator  at  another  site. 

Technical  Support.  MEDCOM  used  monthly  teleconferences  or 
videoconferences  to  communicate  with  the  sites  during  the  demon¬ 
stration.  The  implementation  teams  from  all  the  sites  participated  in 
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the  conferences,  and  participants  discussed  implementation  issues  and 
approaches  to  address  them.  RAND  staff  did  not  participate  in  these 
conferences.  MEDCOM  staff  also  participated  in  the  two  rounds  of 
visits  for  the  RAND  evaluation,  during  which  they  addressed  ques¬ 
tions  from  the  sites  and  more  generally  assisted  them  in  their  imple¬ 
mentation  activities.  During  our  site  visits,  the  MTF  teams  gave 
generally  positive  feedback  about  the  usefulness  of  the  videoconfer¬ 
ences. 


Structure  and  Support  at  the  MTFs 

The  MTF  Environment 

The  four  demonstration  sites  varied  in  their  size  and  clinical  assets,  as 
well  as  in  their  previous  experience  with  quality  improvement  strate¬ 
gies  and  use  of  clinical  practice  guidelines.  These  features,  which  are 
summarized  below,  influenced  the  strategies  chosen  by  the  MTF 
teams  for  implementing  the  asthma  guideline  and  the  actions  they 
undertook  to  carry  out  the  strategies. 

All  the  MTFs  served  a  large  active-duty  population  and  their 
families  and  had  the  basic  clinical  capabilities  for  the  treatment  of 
asthma,  including  primary  care  internal  medicine  clinics.  All  of  them 
had  multiple  family  or  primary  care  clinics  located  in  the  hospital 
and,  in  addition,  two  of  them  ran  troop  medical  clinics  (TMCs)  in 
buildings  separate  from  the  hospital.  At  all  the  MTFs,  a  mix  of  mili¬ 
tary  and  contract  physicians  provided  primary  care  services.  Also,  one 
MTF  had  a  family  practice  residency  program  with  some  30  trainees. 
All  the  MTFs  reported  that  they  had  low  ratios  of  ancillary  staff  to 
providers,  typically  not  exceeding  one  to  one.  Support  staff  limita¬ 
tions  were  a  constraint  on  the  MTFs’  ability  and  willingness  to  take 
on  new  workload  for  implementing  new  practices. 

The  MTFs  differed  in  their  on-site  availability  of  specialty  clin¬ 
ics,  including  respiratory  therapy,  allergist,  and  pulmonologist.  Three 
out  of  the  four  MTFs  already  had  done  some  quality  improvement 
work  for  asthma  care  before  the  DoD/VA  guideline  was  published. 
Two  MTFs  had  previously  sought  or  were  already  implementing  a 
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guideline  for  asthma.  A  third  MTF  was  focusing  on  improving  self- 
care  of  asthma  patients  prior  to  the  demonstration. 

Annual  rotations  and  frequent  deployments  are  inherent  to  the 
Army  environments  and  the  demonstration  sites  were  no  exception. 
All  the  MTFs  experienced  turnover  and  absences  of  key  staff  during 
the  demonstration,  making  it  difficult  to  train  all  providers,  ensure 
that  new  procedures  were  communicated  to  all  personnel  at  the  same 
time,  and  otherwise  provide  continuity  in  the  implementation  of  the 
guideline. 

Baseline  Climate  for  Guideline  Implementation 

As  described  in  Chapter  Two,  a  climate  survey  was  administered  to 
the  command  and  implementation  teams  during  the  kickoff  confer¬ 
ence.  The  climate  survey  consisted  of  four  modules  that  addressed 
motivation  for  guideline  implementation,  supportiveness  of  climate, 
attitudes  toward  practice  guidelines,  and  efforts  to  improve  quality  of 
care  (refer  to  Appendix  C  for  these  items). 

The  baseline  scores  on  the  motivation,  climate,  and  attitudes 
modules  for  the  MTF  command  and  guideline  teams  are  presented  in 
Tables  4.2  and  4.3.  To  protect  privacy,  neither  the  names  of  the 
MTFs  nor  the  number  of  respondents  in  each  MTF  are  provided. 
There  were  a  total  of  61  respondents,  of  whom  17  were  members  of 
MTF  command  teams  and  44  were  on  the  implementation  teams. 
The  teams  ranged  in  size  from  8  to  1 5  members. 

The  command  teams  and  implementation  teams  had  similar 
views  regarding  the  importance  of  all  four  of  the  measures  of  readi¬ 
ness  for  guideline  implementation — the  eight  quality  improvement 
factors,  the  MTFs’  current  status  in  quality  improvement,  the  MTF 
climate,  and  attitudes  toward  practice  guidelines  (Table  4.2).  Both 
groups  gave  moderately  high  scores  to  the  importance  of  quality 
improvement,  with  average  scores  of  34.1  and  35.7  (of  a  maximum  of 
40),  and  for  attitude  toward  practice  guidelines,  with  average  scores 
of  33.2  and  32.6  (of  a  maximum  of  42).  The  relatively  high  scores 
they  gave  to  the  importance  of  improving  quality  and  attitude  toward 
practice  guidelines  suggest  they  were  well  positioned  to  start  working 
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Table  4.2 

Baseline  Survey  Scores  on  Quality  Improvement,  MTF  Climate,  and  Attitudes 
Toward  Practice  Guidelines 


Means  (Standard  Deviations)  for  Views  on  Quality  Activities 


Respondent  Group 

Importance 
of  Improving 
Quality  of 
Care  (Range 

8  to  40) 

MTF  Current 
Status  in 
Quality 
Improvement 
(Range  8  to 
40) 

MTF  Climate 
for  Guideiine 
Impiemen- 
tation  (Range 

7  to  28) 

Attitude 
Toward 
Practice 
Guidelines 
(Range  6 
to  42) 

All  MTFs  (4) 

Command  teams 

34.1  (3.1) 

25.4  (4.4) 

14.7  (2.7) 

33.2  (5.5) 

Implementation 

teams 

35.7  (3.8) 

25.4  (6.1) 

14.6  (4.0) 

32.6  (4.8) 

Combined  by  MTF 

Demo  1 

35.5  (4.1) 

26.5  (5.6) 

14.0  (3.0) 

32.5  (4.6) 

Demo  2 

34.4  (3.8) 

22.0  (5.8) 

13.0  (2.5) 

30.5  (3.0) 

Demo  3 

36.3  (3.8) 

25.8  (4.2) 

15.5  (3.8) 

34.6  (5.8) 

Demo  4 

34.8  (3.0) 

26.8  (5.4) 

16.3  (4.5) 

33.9  (5.7) 

with  the  asthma  guideline.  We  also  found  that  the  command  and 
implementation  teams  at  each  MTF  were  consistent  in  their  ratings 
of  the  MTF  environment. 

A  baseline  motivation  measure  was  derived  for  each  implemen¬ 
tation  team  member  based  on  the  concept  that  team  members  will  be 
motivated  to  initiate  guideline  activities  when  they  perceive  that  their 
efforts  will  lead  to  successful  guideline  implementation,  successful 
implementation  will  lead  to  improved  job  performance,  and 
improved  job  performance  will  be  instrumental  in  obtaining  desired 
outcomes  (e.g.,  career  progress,  improved  patient  outcomes).  We  cal¬ 
culated  motivation  scores  using  survey  responses  for  three  factors: ' 

Motivation  score  =  (effort  yields  performance)  X  (performance  yields 
outcome  factor)  X  (importance  of  outcome  factor) 


*  The  factors  are  measured  as  agreement/disagreement  that  exertion  of  effort  will  help  the 
individual  or  clinic  perform  consistently  with  the  guideline  (scale  of  one  to  seven),  agree¬ 
ment/disagreement  that  their  own  efforts  or  overall  clinic/MTF  efforts  will  contribute  to 
each  quality  improvement  factor  (scale  of  one  to  seven),  and  the  importance  of  each  factor 
(scale  of  one  to  five) . 
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Separate  average  scores  were  calculated  for  individual  motivation 
and  clinic  or  MTF  motivation,  as  well  as  for  a  combined  average 
score  for  overall  motivation.  For  ease  of  interpretation,  these  scores 
were  converted  to  percentages  of  the  maximum  possible  score.  The 
individual  and  combined  motivation  scores  for  the  four  MTFs, 
shown  in  Table  4.3,  varied  from  less  than  46  percent  to  more  than  81 
percent.  Differences  across  MTFs  were  statistically  significant  for  the 
clinic  motivation  and  combined  scores  but  not  for  individual  motiva¬ 
tion.  Considering  the  magnitude  of  the  scores,  the  MTF  teams  were 
moderately  motivated  to  pursue  guideline  implementation,  but  there 
did  not  appear  to  be  overwhelming  enthusiasm  for  the  task,  especially 
for  the  MTF  identified  as  “Demo  2,”  which  had  the  lowest  scores. 

Support  for  the  Demonstration 

As  described  in  Chapter  One,  commanders  of  participating  MTFs^ 
were  asked  to  appoint  a  multidisciplinary  implementation  team  of 
eight  to  ten  individuals  to  develop  an  action  plan  for  the  implementa¬ 
tion  and  monitor  its  progress.  These  individuals  represented  the  mix 


Table  4.3 

Baseline  Motivation  for  Guideline  Implementation  by  the 
Implementation  Teams 


Percentage  of  Maximum  Scores  for  Perceptions 
of  Motivation  by  the  Guideline  Implementation 
Teams® 

Individual 

Clinic/MTF 

Combined 

Military  Treatment  Facility 

Motivation 

Motivation 

Motivation 

Demo  1 

64.1 

66. 1“= 

64.9“= 

Demo  2 

51.4 

45.7‘= 

48.7‘= 

Demo  3 

78.4 

80.0“= 

81.6“= 

Demo  4 

55.1 

62.9‘= 

56.3*= 

®The  index  scores  have  a  maximum  possible  range  of  values  from  1  to  245.  The 
results  reported  are  expressed  as  a  percentage  of  the  maximum  score  of  245. 
Differences  among  the  MTFs  are  not  statistically  significant. 

‘=p<0.05. 


^  The  MTF  commander  is  the  equivalent  of  a  chief  executive  officer  in  a  civilian  hospital. 
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of  clinical  and  support  staff  involved  in  delivering  care  for  patients 
with  asthma. 

The  commanders  also  were  asked  to  designate  a  guideline 
champion  and  a  facilitator.  The  champion  was  the  leader  of  the 
implementation  activities  and  the  MTF  team.  The  facilitator  was  to 
guide  the  implementation  team  in  developing  an  implementation 
action  plan  and  then  to  provide  support  to  the  champion  and  team  in 
coordinating  and  managing  the  implementation  process. 

Through  the  climate  survey  and  site  visits,  we  gathered  informa¬ 
tion  on  the  level  of  support  provided  hy  commanders,  champions, 
facilitators,  and  implementation  teams.  The  climate  survey  assessed 
baseline  attitudes  before  the  implementation  got  under  way,  and  we 
gathered  updated  information  at  the  site  visits  that  we  could  use  to 
assess  how  the  level  of  support  changed  during  implementation. 

Command  Support  and  Accountability.  Staff  at  three  of  the  four 
sites  reported  that  commanders  had  been  supportive  of  the  guideline 
and  its  implementation,  although  this  support  was  passive  in  the 
sense  that  staff  said  that  the  commanders  did  not  demand  regular 
reporting  on  progress,  and  hence  accountability.  Multiple  changes  in 
command  at  one  of  these  sites  did  not  alter  this  positive,  but  passive, 
support.  At  the  fourth  site,  staff  were  ambivalent  as  to  the  extent  of 
command  support,  even  though  command  had  agreed  to  allocate 
funds  to  buy  asthma  pamphlets  for  patient  education  in  addition  to 
those  provided  by  MEDCOM  in  the  asthma  toolkit. 

Climate  surveys  reinforced  these  observations  of  passive  com¬ 
mand  support.  Although  implementation  team  members  responding 
to  the  RAND  survey  perceived  a  “good  chance”  that  a  staff  member 
would  be  noticed  if  he  or  she  did  not  cooperate  with  guideline 
implementation,  they  also  perceived  that  the  staff  member  would  take 
“no  risk”  or  only  “slight  risk”  in  not  cooperating  with  guideline 
implementation.  They  also  perceived  that  “nothing”  to  “very  little” 
would  be  done  if  a  staff  member  were  uncooperative.  Staff  at  two  of 
the  four  sites,  however,  perceived  that  there  would  be  a  “good” 
chance  that  a  staff  member  would  be  praised  for  cooperating  with 
implementation  of  the  asthma  guideline. 
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As  requested,  all  the  commanders  designated  guideline  champi¬ 
ons,  facilitators,  and  implementation  teams  and  they  authorized  the 
teams’  participation  in  the  two-day  off-site  conference  that  initiated 
the  demonstration.  As  communicated  hy  the  MTF  implementation 
teams  in  their  monthly  reports  and  the  evaluation  site  visits,  however, 
when  the  demonstration  activities  began,  none  of  the  MTF  com¬ 
manders  provided  their  champions  or  members  of  their  implementa¬ 
tion  teams  dedicated  time  to  carry  out  the  activities  in  their  guideline 
action  plans.  Team  members  continued  to  be  responsible  for  the 
functions  they  had  to  perform  before  implementation  began,  and  the 
time  spent  implementing  activities  related  to  the  asthma  guideline 
was  added  to  those  responsibilities. 

The  Champions.  The  participating  MTFs  varied  in  their  initial 
choice  of  champions  to  lead  the  asthma  guideline  implementation 
activities,  and  one  site  changed  its  champion  early  in  the  demonstra¬ 
tion.  Two  sites  designated  family  practice  physicians  as  champions, 
one  site  designated  an  internal  medicine  specialist,  and  the  last  site 
designated  the  chief  of  pediatrics.  At  two  of  the  four  sites,  the  cham¬ 
pions  were  civilians,  and  at  the  other  two,  the  champions  were  junior 
officers.  All  champions  reported  during  the  site  visits  that  they  were 
committed  to  the  successful  implementation  of  the  guideline,  and 
other  information  gathered  at  the  site  visits  confirmed  they  had  all 
played  an  active  role  in  training  staff  on  the  guideline. 

However,  during  the  site  visits,  champions  reported  that  the  lack 
of  “protected  time”  allocated  for  implementation  of  the  guideline 
hampered  their  ability  to  be  effective  leaders  of  implementation 
activities.  Overall,  the  champions  reported  that  they  spent  from  1 0  to 
30  percent  of  their  time  in  implementation  activities  of  the  asthma 
guideline.  Some  reported  that  they  did  not  have  the  military  rank  or 
other  professional  status  needed  to  give  them  the  credibility  (informal 
power)  to  persuade  their  colleagues  to  work  with  the  guideline  and  to 
overcome  any  resistance  to  practice  guidelines  that  providers  might 
express. 

The  champion  at  one  site  was  changed  early  in  the  demonstra¬ 
tion.  The  initial  champion  had  played  a  major  role  in  the  develop¬ 
ment  of  the  asthma  guideline  and  felt  he  had  become  a  barrier  in  its 
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implementation.  His  high  expectations  that  his  colleagues  would 
share  his  level  of  knowledge  of  the  guideline  and  commitment  to 
implementing  it  were  leading  to  miscommunication  and  frustrations. 
He  voluntarily  relinquished  his  role  to  two  of  his  colleagues,  a  phar¬ 
macist  and  a  family  practice  physician. 

The  Facilitators.  The  demonstration  MTFs  selected  facilitators 
with  a  variety  of  backgrounds  to  support  MTF  teams  in  their  plan¬ 
ning  and  execution  of  implementation  actions.  Two  of  the  facilita¬ 
tors,  one  civilian  and  one  military,  were  registered  nurses  assigned  to 
utilization  management,  one  facilitator  was  a  military  nurse  case 
manager,  and  the  fourth  was  a  civilian. 

The  extent  of  involvement  of  the  facilitators  in  implementation 
activities  varied  across  sites.  At  one  site,  the  facilitator  actively  led  the 
revisions  of  forms  and  the  development  of  new  procedures  for  patient 
flows.  Later,  she  pulled  hack  to  take  a  role  of  “ohjective”  observer 
providing  feedback  on  implementation  progress.  Another  facilitator 
played  a  major  role  in  generating  data  to  monitor  progress  made  on 
the  asthma  metrics.  The  other  two  facilitators  played  passive  roles, 
leaving  the  coordination  of  activities  to  the  champions. 

The  Implementation  Teams.  Drawing  on  previous  experience 
with  group  processes,  MEDCOM  and  RAND  advised  the  MTFs  to 
establish  multidisciplinary  teams  of  8  to  11  members.  This  is  based 
on  published  findings  from  experience  with  team  management  that 
suggest  that  too  few  or  too  many  team  members  reduce  team  effec¬ 
tiveness,  with  optimal  size  ranging  from  6  to  10  members  (Cohen 
and  Bailey,  1997;  Starfield,  1998). 

Three  of  the  MTFs  complied  with  this  guidance,  establishing 
teams  of  8  to  12  members.  The  fourth  MTF  initially  chose  a  16- 
member  team  because  it  wanted  to  include  one  representative  or 
more  from  all  of  its  multiple  family  and  specialty  clinics.  It  later 
reduced  the  size  of  the  team  to  13  members,  including  four  civilians 
that  the  site  felt  would  provide  continuity  over  time.  The  size  was 
reduced  because  they  found  the  larger  team  was  difficult  to  manage 
and  not  productive.  The  membership  of  the  teams  remained  rela¬ 
tively  constant  during  the  1 5  months  between  the  kickoff  conference 
and  our  last  site  visit. 
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With  a  few  exceptions,  the  implementation  teams  included  the 
clinical  and  support  staff  appropriate  to  the  implementation  of  the 
asthma  guideline:  primary  care  providers,  pediatricians,  pharmacists, 
internists,  respiratory  therapists,  nurses,  ancillary  staff,  and  utilization 
and  quality  managers.  One  site  included  an  allergist.  A  notable  excep¬ 
tion  was  a  site  that  did  not  include  a  pharmacist  on  its  implementa¬ 
tion  team,  reportedly  because  pharmacy  operates  separately  from  the 
MTF.  Against  MEDCOM  and  RAND  advice,  two  sites  did  not 
include  a  representative  from  ancillary  staff  on  their  implementation 
teams.  Failure  to  do  so  resulted  in  resistance  by  the  ancillary  staff  to 
adopting  practice  changes  that  required  their  constructive  participa¬ 
tion,  which  was  reported  to  us  by  the  MTF  teams  and  those  staff 
during  the  site  visits. 

In  spite  of  staff  continuity  and  autonomy,  most  of  the  members 
of  the  implementation  teams  were  minimally  involved  in  managing 
the  actual  implementation  process  for  the  asthma  guideline.  Imple¬ 
mentation  team  members  reported  that  the  lack  of  dedicated  time 
allocated  to  members  of  the  implementation  teams  was  one  reason  for 
their  low  involvement.  Another  contributing  factor  was  that  many  of 
the  key  implementation  actions,  such  as  training  of  staff  or  changes 
made  to  forms,  were  primarily  handled  by  the  champions  or  facilita¬ 
tors,  which  reduced  the  sense  of  “ownership”  in  the  implementation 
process  by  other  team  members. 

This  low  participation  by  other  MTF  team  members  is  reflected 
in  patterns  of  team  meetings.  Two  sites  held  no  regular  meetings,  and 
the  few  that  were  held  were  poorly  attended.  The  other  two  sites  ini¬ 
tially  held  regular  weekly  to  biweekly  meetings  that  eventually  were 
reduced  in  frequency  to  once  a  month.  One  site  created  an  e-mail 
alias  to  facilitate  communications  among  members  of  its  implementa¬ 
tion  team. 

As  a  result,  the  primary  role  of  the  implementation  team  mem¬ 
bers  was  to  provide  across-the-MTF  support  and  legitimacy  for 
implementation  of  the  asthma  guideline.  Utilization  management 
and  quality  management  members  of  the  implementation  teams  at 
three  sites  were  exceptions,  having  devoted  considerable  time  to 
generating  monitoring  reports  on  the  asthma  metrics. 


68  Implementation  of  the  Asthma  Practice  Guideline  in  AMEDD 


Implementation  Activities  and  Progress 

Although  all  sites  were  encouraged  to  start  implementation  of  their 
action  plans  soon  after  returning  from  the  kickoff  conference,  only 
two  sites  actually  did.  The  other  two  sites  did  not  begin  implementa¬ 
tion  until  five  months  after  the  kickoff  conference.  At  one  of  these 
sites,  a  two-month  deployment  and  provider  resistance  to  the  guide¬ 
line  forced  a  delay  in  start-up  activities.  In  response  to  these  delays, 
we  adjusted  our  site  visit  schedule  so  that  the  first  site  visits  were  held 
in  February  and  March  2000  instead  of  in  January,  which  had  origi¬ 
nally  been  planned,  with  the  second  site  visits  six  months  later.  We 
describe  below  the  implementation  strategies  and  activities  of  the 
demonstration  sites,  and  we  discuss  the  factors  that  appear  to  have 
affected  their  progress. 

Implementation  Strategies 

The  action  plans  developed  by  the  MTFs  at  the  kickoff  conference, 
and  subsequently  updated  as  they  gained  implementation  experience, 
generally  addressed  all  four  modules  of  the  guideline:  initial  diagnosis, 
follow-up  visits,  emergency  management  of  asthma  exacerbations, 
and  telephone  triage.  However,  our  review  of  the  written  plans  con¬ 
firmed  that  specific  actions  tended  to  be  focused  on  the  first  two 
modules.  All  plans  included  activities  to  introduce  the  guideline  and 
forms  to  both  providers  and  ancillary  staff,  to  increase  patient  educa¬ 
tion  and  case  management,  and  to  purchase  spirometers  or  make 
them  more  readily  accessible  to  patients. 

All  MTFs  identified  the  metrics  they  planned  to  monitor  on  a 
monthly  or  quarterly  basis  using  the  ADS,  CHCS,  or  chart  reviews 
(see  Table  4.4),  and  they  planned  to  begin  monitoring  progress  early 
and  regularly.  In  addition,  one  site  planned  to  develop  a  registry  of 
current  asthma  patients.  All  sites  aimed  to  complete  implementation 
of  all  their  planned  actions  MTF-wide  within  seven  to  nine  months 
of  the  kickoff  conference. 

All  of  the  metrics  selected  by  the  MTFs  directly  address  impor¬ 
tant  aspects  of  care  as  defined  in  the  practice  guideline,  and  data  for 
all  of  them  require  data  from  the  medical  chart  of  MTF  data  systems 
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to  measure.  These  data  are  not  available  to  MEDCOM  because 
AMEDD  does  not  have  electronic  medical  records  and  the  other 
MTE  data  systems  are  not  linked  in  a  network.  It  is  for  these  reasons 
that  these  measures  are  different  from  the  indicators  RAND  mea¬ 
sured  in  our  assessment  of  effects  of  guideline  implementation.  We 
were  limited  to  indicators  that  could  be  measured  using  encounter 
data  and  centrally  available  data  on  use  of  prescription  drugs. 

The  draft  action  plans  gave  limited  attention  to  specific  actions 
to  change  the  clinics’  existing  structures  or  procedures.  The  MTEs 
indicated  they  would  use  the  MEDCOM  asthma  forms,  improve 
patients’  access  to  the  clinics,  or  develop  a  system  for  follow-up  visits. 
In  addition,  our  review  found  that  most  of  the  specific  actions  out¬ 
lined  in  the  MTEs’  plans  were  conceived  as  onetime  actions  to  be 
taken  outside  of  the  ongoing  education  and  managerial  processes  of 
the  MTEs  and  their  related  clinics  and  TMCs. 

At  the  kickoff  conference,  the  teams  were  encouraged  to 
approach  implementation  by  undertaking  actions  in  one  clinic  first. 


Table  4.4 

Asthma  Metrics  Initially  Selected  by  the  Demonstration  Sites 


Fort 

Fort 

Fort 

Fort 

Metrics 

Benning 

Campbell 

Gordon 

Jackson 

Percentage  of  patients  with  docu¬ 
mented  severity 

Percentage  of  patients  initially 

X 

X 

X 

identified  who  were  referred  for 
spirometry 

X 

X 

X 

Percentage  of  patients  with  spi¬ 

rometry  in  last  12  months 

X 

X 

Percentage  of  patients  with  written 
action  plan 

Percentage  of  patients  with  medi¬ 

X 

X 

X 

cations  appropriate  to  level  of 
severity 

X 

X 

X 

X 

Percentage  of  target  group  not 
diagnosed  appropriately 

Percentage  of  inappropriate  refer¬ 

X 

rals  to  emergency  center  in  tele¬ 
phone  triage 

X 

Frequency 

Quarterly 

Monthly, 

Quarterly 

Monthly 

Monthly 
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through  which  they  could  gain  experience  and  correct  prohlems  iden¬ 
tified  before  launching  a  major  change  in  practices  in  all  clinics  and 
TMCs.  Three  of  the  sites  used  this  approach,  beginning  implementa¬ 
tion  in  one  clinic  and  planning  to  expand  to  implementation  across 
all  MTF  clinics  that  served  asthma  patients  within  four  months.  The 
fourth  site  already  had  been  using  an  asthma  guideline  before  the 
demonstration,  and  it  continued  its  practices,  expanding  its  patient 
education  and  disease  management  activities. 

In  general,  all  the  MTFs  followed  their  action  plans  as  they 
began  implementation,  although  two  exceptions  are  worth  noting. 
One  MTF,  demonstration  site  1,  when  faced  with  provider  resistance, 
scaled  down  its  original  comprehensive  approach  to  focus  on  just  two 
metrics  of  the  guideline.  Second,  none  of  the  MTFs  implemented 
their  intended  changes  to  their  telephone  triage  practices.  Three 
MTFs  never  addressed  this  issue  during  the  12-month  duration  of  the 
demonstration.  The  fourth  MTF  reported  that  it  had  tried  to  change 
the  ways  its  telephone  triage  contractor  advised  asthmatic  callers,  but 
its  efforts  had  been  unsuccessful.  Eventually,  it  developed  its  own 
protocol  for  use  in  telephone  triage,  but  it  had  not  yet  implemented 
the  protocol  by  the  end  of  the  demonstration.^ 

Demonstration  Site  1.  Initially,  this  MTF  sought  to  implement 
comprehensive  actions  to  address  each  element  of  the  guideline, 
including  initial  diagnosis,  ongoing  management,  emergency  treat¬ 
ment  and  telephone  triage.  This  approach  met  with  strong  resistance 
from  providers.  In  addition,  a  lengthy  two-month  deployment 
immediately  following  the  kickoff  conference  caused  a  loss  of 
momentum.  To  address  these  issues,  the  implementation  team  wrote 
a  new  action  plan  that  focused  on  just  two  dimensions  of  the  guide¬ 
line:  identification  of  level  of  severity  and  appropriateness  of  treat¬ 
ment  for  the  level  of  severity  diagnosed.  To  support  this  new 
approach,  which  it  saw  as  a  first  step  in  fully  implementing  the 


^  We  note  that  such  adjustments  to  implementation  action  plans  are  to  be  expected  as  orga¬ 
nizations  experience  successes  with  some  actions  and  barriers  to  carrying  out  others.  For  this 
reason,  we  encouraged  the  MTFs  to  adjust  their  plans  accordingly,  while  remaining  focused 
on  the  goal  of  increasing  consistency  with  the  provisions  of  the  asthma  guideline. 
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guideline,  the  site  relied  on  clinic-by-clinic  monitoring  of  compli¬ 
ance.  The  site  also  sought  to  increase  referrals  of  patients  to  its 
Asthma  Resource  Center,  which  it  had  established  prior  to  the  dem¬ 
onstration  to  provide  patient  education.  By  the  end  of  the  demonstra¬ 
tion,  the  site  had  implemented  this  first  increment  of  the  guideline  in 
all  of  its  clinics  except  pediatrics  but  had  not  yet  begun  to  address  the 
other  elements  of  the  guideline  in  all  clinics. 

Demonstration  Site  2.  This  site  started  implementation  of  the 
asthma  guideline  in  its  family  practice  clinic,  its  largest  clinic  and 
home  of  the  family  practice  residence  program.  In  the  words  of  the 
staff  at  the  MTF,  reported  at  our  site  visit,  the  site  took  a  “minimal¬ 
ist”  approach,  simply  distributing  the  asthma  key  elements  cards  and 
making  the  MEDCOM  forms  available.  When  monitoring  showed 
low  compliance  rates  with  documentation  of  asthma  care,  the  site 
provided  more  thorough  training  to  both  providers  and  ancillary 
staff  Experience  gained  in  this  pilot  program  led  the  site  to  establish 
a  case  management  capability  that  focused  on  formalizing  referrals  of 
acute  asthma  patients  by  PCMs  to  asthma  education  and  from  the 
emergency  room  to  the  PCMs.  The  site  also  revised  the  content  of  its 
asthma  education  classes  and  scheduled  them  regularly,  and  it  devel¬ 
oped  a  list  of  acute  asthma  patients  to  refer  for  case  management. 
Chart  audits  were  begun  early  in  the  demonstration  to  guide  imple¬ 
mentation. 

Implementation  of  the  guideline  at  the  site’s  internal  medicine 
clinic,  emergency  room,  and  its  numerous  TMCs  began  in  the  sev¬ 
enth  month  of  the  demonstration,  three  months  later  than  planned. 
Reportedly,  implementation  of  the  guideline  in  the  TMCs  was 
greatly  facilitated  by  the  inclusion  of  the  physician  assistants’  coordi¬ 
nator  on  the  site  implementation  team. 

Demonstration  Site  3.  This  site  used  the  demonstration  as  an 
opportunity  to  reinforce  its  focus  on  patient  education  and  case  man¬ 
agement  of  acute  asthma  cases.  On  ascertaining  that  the  asthma 
guideline  it  had  developed  two  years  prior  to  the  demonstration  was 
consistent  with  the  DoD/VA  guideline,  it  decided  to  continue  to  use 
its  own  guideline  and  forms.  In  its  action  plan,  it  defined  actions  to 
strengthen  referrals  of  patients  to  its  asthma  center,  which  the  site  had 
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established  to  provide  continuity  of  care  through  case  management 
and  patient  education  more  focused  and  effective  than  physicians 
could  provide.  However,  the  center  was  reportedly  chronically  under¬ 
staffed  and  suffered  from  regular  turnover  of  case  managers,  so  it  was 
difficult  for  the  center  to  respond  to  increased  demand  for  its  services. 
The  site  also  developed  a  list  of  asthma  patients  that  it  used  to  iden¬ 
tify  patients  with  a  large  number  of  visits  for  referral  to  case  manage¬ 
ment. 

The  MTF’s  asthma  guideline  was  already  in  use  at  all  of  its 
clinics  and  TMCs,  although  with  varying  degrees  of  success.  Imple¬ 
mentation  in  the  TMCs  was  made  more  difficult  because  TMC  staff 
reports  to  division  command,  rather  than  the  MTF  commander  and 
because  physician  assistants  are  reluctant  to  mark  asthma  as  a  diagno¬ 
sis  because  they  perceive  it  may  jeopardize  soldiers’  careers.  By  the 
end  of  the  demonstration,  the  site  had  not  begun  to  monitor  the 
asthma  metrics. 

Demonstration  Site  4.  A  year  before  the  demonstration,  this  site 
had  sought  to  improve  the  treatment  of  asthma  patients  by  empha¬ 
sizing  self-care  and  the  promotion  of  a  patient  action  plan.  With 
implementation  of  the  guideline,  initially  at  its  family  health  clinic, 
the  site  formalized  referrals  to  patient  education  and  to  spirometry  to 
ensure  timely  reporting  of  results  to  providers.  It  also  began  to  moni¬ 
tor  the  asthma  metrics  early  in  the  demonstration  and  developed  a  list 
of  asthma  patients.  The  site  delayed  implementation  of  the  guideline 
to  its  internal  medicine  clinic  to  avoid  high  turnover  of  staff  during 
the  summer. 

The  Implementation  Process  and  Activities 

To  carry  out  their  respective  strategies,  the  sites  were  asked  to  intro¬ 
duce  the  guideline  algorithm  and  supporting  toolkit  items  to  provid¬ 
ers  and  staff,  make  changes  to  administrative  procedures,  provide 
patient  education  and  self-management,  and  monitor  selected  indica¬ 
tors.  The  experiences  of  the  four  sites  in  each  of  these  implementation 
steps  are  compared  below. 

Guideline  Introduction  and  Training.  All  the  sites  began  their 
implementation  activities  by  holding  education  sessions  for  primary 
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care  providers  to  introduce  them  to  the  evidence-hased  practices 
specified  in  the  asthma  guideline.  Two  of  the  sites  gave  group  train¬ 
ing  sessions  that  provided  continuing  medical  education  (CME) 
credits  while  the  other  two  provided  training  during  a  regular  staff 
meeting.  These  initial  sessions  did  not  reach  all  relevant  staff  because 
of  deployments  and  work  schedule  conflicts.  At  one  site,  60  percent 
of  providers  participating  in  our  focus  groups  said  they  had  not 
attended  a  training  session  on  the  guideline.  One  site  reported  that 
staff  at  the  TMCs  were  particularly  difficult  to  reach,  not  only 
because  of  frequent  deployments  and  rotations  but  also  because  staff 
reported  to  their  field  units.  Another  site  also  indicated  that  contract 
and  resource  staff  had  incentives  that  discouraged  participation  in 
training.  Contractually,  these  providers  are  paid  by  the  quantity  of 
services  they  provide,  and  time  spent  in  training  is  a  diversion  from 
this  activity. 

The  content  of  the  training  sessions  varied  among  the  sites.  Two 
sites  centralized  training  and  used  the  PowerPoint  presentation 
included  in  the  toolkit  and  available  on  the  MEDCOM  Quality 
Management  web  site.  One  of  these  sites  also  provided  a  handout 
from  the  NHEBI  in  the  hope  that  it  would  increase  provider  accep¬ 
tance  of  the  DoD/VA  asthma  guideline.  Another  site  used  its  own 
educational  material.  This  site  had  already  conducted  educational 
programs  on  the  appropriate  treatment  of  asthma  using  the  NHEBI 
guideline.  It  complemented  this  training  with  CME  lectures,  individ¬ 
ual  meetings  between  providers  and  case  managers,  and  academic 
detailing  conducted  by  the  asthma  center  staff 

Of  the  providers  at  these  three  sites  who  participated  in  our 
focus  groups,  90  percent  had  received  a  copy  of  the  full  asthma 
guideline  and  a  copy  of  the  laminated  sheet  showing  its  key  elements. 
The  same  proportion  had  seen  the  MEDCOM  encounter  form  701- 
R  and  two-thirds  had  seen  a  copy  of  the  MEDCOM  patient  action 
plan  form  703-R.  Eorm  702-R  received  no  comment.  Providers  who 
had  participated  to  the  training  sessions  rated  the  training  received 
“somewhat  helpful”  on  the  average. 

The  fourth  site  took  a  very  different  approach  to  staff  training 
after  providers  in  this  hospital  exhibited  considerable  resistance  to 
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accepting  the  DoD/VA  asthma  guideline.  Its  philosophy  was  to 
require  providers  to  use  the  guideline  first  and  then  to  educate  them. 
To  overcome  staff  resistance,  the  training  emphasized  that  imple¬ 
menting  evidence-hased  clinical  guidelines  was  a  priority  of  the  Army 
Surgeon  General.  It  also  emphasized  that  providers  were  expected  to 
document  their  diagnosis  on  the  patient  encounter  form  (701-R)  and 
to  have  an  asthma  action  plan  documented  on  form  703-R  for  every 
patient.  It  was  made  clear  that  the  focus  of  the  groups’  discussion  was 
to  he  on  how  to  implement,  not  on  whether  to  implement.  The  cham¬ 
pion  provided  this  training  in  each  individual  clinic.  They  had  gener¬ 
ally  active  involvement  hy  the  providers,  in  part  because  the  MTF 
team  made  it  clear  that  the  providers  had  to  address  the  practice 
guideline. 

Having  learned  from  the  earlier  low  hack  pain  demonstration 
that  failure  to  train  the  ancillary  staff  led  to  reluctant  cooperation, 
MEDCOM  emphasized  at  the  kickoff  conference  the  importance  of 
introducing  the  asthma  guideline  to  the  sites’  ancillary  and  support 
staff  and  of  clearly  defining  their  roles  in  the  implementation  process. 
Two  sites  heeded  this  advice  and  gave  separate  formal  training  ses¬ 
sions  to  their  clinic  staff.  The  first  site  provided  separate  sessions  for 
nurses,  clerks,  and  other  support  staff.  The  content  of  the  training 
sessions  was  developed  hy  respiratory  therapy  staff,  and  a  nurse  con¬ 
ducted  the  sessions.  The  other  site  trained  ancillary  staff  in  a  regularly 
scheduled  staff  meeting.  The  other  two  sites  did  not  formally  intro¬ 
duce  the  asthma  guideline  to  their  ancillary  staff  other  than  showing 
them  MEDCOM  form  701-R  and  instructing  them  to  make  it  avail¬ 
able  in  the  visit  rooms. 

tVII  sites  recognized  that  they  needed  to  offer  additional  and 
periodic  training  sessions  for  existing  staff  and  newcomers.  Because 
variations  in  work  schedules  and  frequent  deployments  and  rotations 
of  staff  at  all  levels  are  a  fact  of  life  in  the  military,  some  of  the  sites 
had  given  some  thought  to  integrating  periodic  training  on  the 
asthma  guideline  into  orientation  sessions  for  newcomers.  As  of  the 
end  of  the  demonstration,  none  of  the  sites  had  yet  done  this.  Sites 
continued  to  rely  on  training  “on  the  job”  by  peers  and,  at  one  site, 
on  individual  training  provided  by  asthma  center  case  managers. 
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Procedures  for  Patient  Visits.  To  ensure  documentation  of 
asthma  diagnosis  and  treatment,  all  sites  initially  sought  to  encourage 
providers  to  use  the  MEDCOM  encounter  form  701-R  or,  at  one 
site,  its  local  equivalent.  Compliance  tended  to  he  low  at  all  sites  and 
varied  depending  on  the  availability  of  ancillary  staff  support,  primary 
care  provider  acceptance  of  the  form,  and  aggressiveness  of  monitor¬ 
ing. 

At  three  sites,  the  approach  was  to  identify  an  asthma  patient  at 
the  front  desk  or  in  the  vital  signs  room  and  to  ask  the  patient  to  fill 
out  the  patient  portion  of  encounter  form  701-R.  At  one  of  these 
sites,  form  701-R  was  attached  to  the  patient  chart  when  charts  were 
pulled  out  in  the  morning  for  the  day’s  appointments.  In  this  case, 
the  patient  was  asked  to  fill  out  the  form  when  signing  in  at  the  desk. 
At  another  of  these  sites,  posters  were  placed  above  the  sign-in  desk  to 
remind  incoming  asthma  patients  to  identify  themselves.  The 
approach  at  the  fourth  site  similarly  sought  to  identify  asthma 
patients  at  the  front  desk,  and  asthma  was  stamped  on  the  R-600 
form  for  those  patients.  The  form  701-R  then  was  placed  in  the  exam 
room  for  the  providers  to  fill  out.  Levels  of  compliance  with  these 
procedures  varied  from  no  compliance  to  about  50  percent  compli¬ 
ance.  Providers  participating  in  our  focus  groups  varied  widely  in 
their  reported  use  of  the  form. 

A  variety  of  reasons  were  given  for  low  use  of  the  encounter 
form.  MEDCOM  had  given  direction  that  the  form  was  meant  to 
provide  a  convenient  and  efficient  way  to  document  asthma  diagnosis 
and  treatment,  but  that  its  use  was  voluntary,  so  long  as  diagnosis  and 
treatment  were  documented  in  some  fashion  in  the  patient  chart.  The 
sites  left  it  to  individual  providers  to  decide  whether  they  would  use 
the  form.  One  site  used  what  it  characterized  as  a  “low  intensity” 
approach  to  encouraging  providers  to  comply  with  the  guideline,  and 
to  use  form  701-R  in  particular.  This  site  gave  providers  occasional 
reminders  as  well  as  feedback  on  practices  using  the  results  of  anony¬ 
mous  chart  audits. 

Many  ancillary  staff  perceived  that  use  of  the  documentation 
form  would  add  to  an  already  heavy  workload  and,  hence,  were  reluc¬ 
tant  to  use  it.  At  one  site,  the  ratio  of  support  staff  to  ancillary  staff 
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was  as  low  as  one  to  two.  Feeling  already  overburdened,  the  ancillary 
staff  had  little  motivation  to  ensure  that  the  form  was  fdled  out  and 
attached  to  the  chart,  especially  if  it  required  any  allocation  of  their 
time  to  assist  patients  in  filling  it  out.  Because  of  ancillary  staff  resis¬ 
tance  to  the  form,  one  site  decided  to  bypass  this  staff  and  another 
site  discontinued  use  of  the  form  toward  the  end  of  the  demonstra¬ 
tion. 

Other  reasons  for  low  use  of  form  701-R  (or  its  equivalent)  were 
providers’  dislike  of  a  checklist  form,  issues  regarding  which  form  to 
use  for  patients  with  multiple  conditions,  and  proliferation  of  forms. 
Checklist  forms  are  contrary  to  the  long-standing  tradition  in  which 
physicians  document  care  by  writing  ftee-text  notes  on  a  generic 
encounter  form  in  the  style  they  prefer.  For  patients  with  multiple 
conditions,  the  use  of  condition-specific  forms  raised  practical  ques¬ 
tions  about  which  form(s)  should  take  precedence.  Finally,  the 
growing  number  of  condition-specific  forms  to  fill  out  was  a  general 
concern  at  all  sites.  The  sites  encountered  another  issue  in  working 
with  the  encounter  form,  which  involved  placement  of  the  form  in 
the  patient  chart.  Their  understanding  was  that  AMEDD  regulations 
require  that  only  the  SF-600  forms  can  be  placed  in  the  patient  chart, 
and  the  forms  are  to  be  in  chronological  order.  Some  providers 
expressed  frustration  at  having  to  search  for  the  form  in  the  chart. 
The  instructions  for  Form  701-R  stated  that  this  form  also  was  to  be 
placed  in  the  chart  in  chronological  order,  but  MTF  staff  were  not 
aware  of  this  instruction. 

Standardization  of  ADS  Reporting.  By  the  end  of  the  demon¬ 
stration,  two  sites  had  successfully  standardized  coding  of  asthma 
cases  using  the  ICD-9  code  493.1  that  MEDCOM  had  established. 
These  sites  also  had  added  a  field  for  level  of  severity  on  the  ADS 
entry  form  and  were  using  it.  Neither  of  the  other  two  sites  was  cod¬ 
ing  asthma  on  the  ADS  forms  as  had  been  specified  by  MEDCOM. 

Three  of  the  sites  provided  several  reasons  to  explain  why 
appropriate  coding  continued  to  be  an  issue.  Eirst,  coding  is  time- 
consuming,  so  it  is  often  delayed  and  prone  to  errors.  One  site  that 
checked  the  ICD-9  coding  of  its  asthma  encounters  found  inappro¬ 
priate  coding  in  nearly  50  percent  of  cases.  Second,  several  sites  sug- 
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gested  that  some  providers  hesitated  to  code  a  soldier  or  family 
member  as  having  asthma  for  fear  that  this  information  could  he  det¬ 
rimental  to  the  soldier’s  career.  The  issue  appears  to  he  particularly 
salient  at  TMCs  where  physician  assistants  tend  to  code  asthma 
patients  as  having  “reactive  airway  disease,”  reportedly  because  of  the 
stigma  attached  to  asthma  for  active-duty  soldiers.  Another  set  of 
issues  was  found  for  basic  trainees.  One  site  suggested  that  the  incen¬ 
tives  for  recruiters  to  sign  up  new  personnel  makes  them  less  vigilant 
in  identifying  such  conditions  as  asthma  that  would  prevent  an  oth¬ 
erwise  enthusiastic  candidate  from  enlisting. 

Formalized  Referrals  to  Spirometry.  At  the  beginning  of  the 
demonstration,  two  sites  experienced  long  waits  for  spirometry  read¬ 
ings  and  the  return  of  results.  One  site  resolved  this  issue  by  devel¬ 
oping  a  consultation  form  and  implementing  a  procedure  for  referrals 
from  providers  to  the  pulmonary  clinic.  By  the  end  of  the  demonstra¬ 
tion,  spirometry  was  completed  on  the  same  day  a  patient  was 
referred.  The  other  site  addressed  its  access  problem  by  obtaining  a 
grant  that  allowed  it  to  purchase  five  spirometers. 

Patient  Education.  All  four  sites  reported  that  they  viewed 
patient  education  as  an  important  component  of  the  management  of 
asthma,  but  the  sites  varied  greatly  in  the  relative  emphasis  they 
placed  on  implementing  education  activities.  They  all  had  formalized 
procedures  for  providers  to  use  in  referring  patients  to  patient  educa¬ 
tion,  but  the  provider  had  the  discretion  of  whether  or  not  to  make 
such  referrals.  The  implementation  teams  of  all  the  MTFs  viewed 
provision  of  separate  patient  education  services  as  a  way  to  partially 
relieve  providers  of  this  responsibility,  but  it  would  require  that  pro¬ 
viders  be  willing  to  make  the  referrals  to  those  services. 

One  site  had  made  patient  education  the  core  of  its  approach  to 
management  of  asthma  patients  even  before  the  implementation  of 
the  guideline.  In  1998,  this  site  allocated  considerable  resources  to  set 
up  an  asthma  center  in  its  primary  care  clinic,  which  was  staffed  by  a 
respiratory  therapist,  a  nurse  case  manager,  and  temporary  staff 
awaiting  reassignment  on  “medical  hold.”  The  center  provided  group 
and  individual  education  in  asthma  self-management  as  well  as  case 
management  services,  including  proactive  screening  of  emergency 
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room  visits  by  asthmatic  patients  and  screening  of  all  asthma  diag¬ 
nostic  codes  to  identify  patients  with  high  utilization  of  health-care 
resources.  Automated  consultations  for  patients  referred  to  the  center 
were  sent  through  the  CHCS.  Both  the  center  staff  and  providers 
report  that  this  automated  system  works  well.  The  center’s  staff 
makes  appointments  for  patients  through  clinic  clerks,  and  they  fol¬ 
low  up  with  patients  they  have  not  seen  in  a  while. 

Another  site  established  an  asthma  resource  center  in  1 997.  This 
center  provided  education  in  self-management  exclusively.  To  further 
encourage  referrals  to  patient  education,  this  site  developed  an 
appointment  form,  and  clinic  clerks  were  instructed  to  direct  patients 
to  the  center  to  schedule  an  appointment.  In  addition,  a  formal  pro¬ 
cedure  using  the  CHCS  was  put  in  place  at  the  emergency  room 
through  which  asthma  patients  coming  for  emergency  care  were 
referred  both  to  his  or  her  primary  care  physician  for  clinical  care  and 
to  the  center  for  asthma  education  services. 

Patient  education  at  the  other  two  sites  was  less  visible  and  not 
as  well  resourced.  A  clinic  nurse  provided  patient  education  at  these 
two  sites  as  one  component  of  his  or  her  duties.  At  one  site,  respira¬ 
tory  therapy  provided  limited  education  on  the  use  of  peak  flow 
meters,  inhalers,  and  spacers  but  more  comprehensive  asthma  educa¬ 
tion  was  not  included. 

The  length  and  content  of  the  education  courses  and  the  mate¬ 
rial  used  for  education  in  asthma  self-management  varied  among  the 
sites.  All  sites  provided  group  classes,  and  three  sites  also  provided 
individual  classes.  One  site  required  all  family  members  to  attend 
classes,  especially  if  the  patient  was  a  child.  The  classes  varied  in  dura¬ 
tion  from  30  to  90  minutes.  At  one  site,  the  class  was  limited  to 
teaching  the  use  of  peak  flow  meters,  with  no  other  aspect  of  disease 
self-management  or  inhaler  technique  covered.  Another  site  also  cov¬ 
ered  a  number  of  other  topics,  including  anatomy,  physiology,  medi¬ 
cations,  and  triggers. 

tVII  but  one  site  used  their  own  educational  materials  that  they 
had  developed  over  the  years.  All  sites  used  both  written  and  visual 
educational  tools,  some  of  which  were  made  available  to  patients  to 
take  home.  As  noted  earlier,  few  sites  used  the  patient  education 
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material  provided  in  the  MEDCOM  asthma  toolkit,  with  the  excep¬ 
tion  of  the  patient  brochure  developed  hy  the  Red  Cross.  Two  sites 
used  MEDCOM  form  704-R  in  its  education  classes. 

Despite  these  efforts,  referrals  to  education  were  reported  to  he 
low  and  only  about  50  percent  of  scheduled  patients  were  reported  to 
actually  attend.  Some  providers  speculated  that  a  reason  for  low 
attendance  in  patient  education  was  that  patients  do  not  want  to  take 
responsibility  for  their  care.  Some  physicians  were  also  skeptical  about 
the  effectiveness  of  patient  self-management  education.  They  were 
also  concerned  that  they  did  not  receive  feedback  about  whether 
referred  patients  had  actually  attended  classes  and,  if  they  did,  what 
information  they  had  acquired.  Even  when  one  site  had  demonstrated 
that  patients  given  education  had  fewer  emergency  and  clinic  visits 
and  fewer  hospitalizations,  providers  remained  unconvinced. 

One  possible  reason  for  provider  resistance  could  be  lack  of  con¬ 
fidence  in  the  education  services  being  offered  by  the  MTEs.  Skepti¬ 
cism  apparently  was  fueled  by  the  education  programs’  tendency  to 
use  one-way,  didactic  patient  education  rather  than  collaborative, 
problem-solving  approaches  that  providers  feel  are  more  effective. 
Patient  educators  at  the  sites  demonstrated  varying  levels  of  under¬ 
standing  of  the  difference  between  these  two  educational  approaches 
and  generally  followed  the  more  didactic  approach.  None  of  the  sites 
engaged  the  patient  in  the  self-care  process  through  collaborative  goal 
setting  with  patients  or  followed  up  with  patients  on  their  success  in 
meeting  their  goals. 

Case  Management.  Two  of  the  sites  had  case  management 
capabilities  in  place  prior  to  the  demonstration  and  continued  to  refer 
patients  to  it.  A  third  site,  concerned  about  continuity  of  care,  desig¬ 
nated  a  nurse  to  provide  case  management  as  a  result  of  the  demon¬ 
stration.  Patients  making  frequent  visits  (at  one  site  defined  as  12  or 
more  visits  per  year)  were  referred  for  case  management.  At  two  sites, 
asthma  patients  seen  in  the  emergency  room  and  inpatients  were  also 
considered  for  case  management. 

Referrals  from  the  Emergency  Room.  Three  sites  put  in  place 
formalized  referral  procedures  for  asthma  patients  visiting  the  MTE 
emergency  room.  Patients  at  one  site  were  automatically  referred  to 
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their  PCM  and  also  to  the  asthma  center  for  patient  education  while, 
at  another  site,  patients  were  referred  only  to  their  primary  care  man¬ 
ager.  At  the  third  site,  patients  were  referred  only  to  the  asthma  cen¬ 
ter  for  patient  education. 

List  of  Asthma  Patients.  The  sites  had  been  encouraged  at  the 
kickoff  conference  to  develop  a  list  of  their  asthma  patients.  All  sites 
developed  these  lists,  hut  they  conceded  that  the  lists  were  incomplete 
and  contained  many  errors.  The  sites  used  the  lists  to  identify  patients 
with  high  frequency  of  visits  for  referrals  for  case  management.  No 
site  made  its  list  available  to  the  clinics  to  assist  in  the  identification 
of  incoming  patients  with  asthma. 

Monitoring  and  Feedback.  All  sites  planned  to  monitor  the 
MEDCOM  asthma  metrics  on  a  monthly  to  quarterly  basis.  Three  of 
the  sites  actually  performed  monitoring  using  chart  reviews.  They 
completed  two  rounds  of  chart  reviews  within  three-  to  six-month 
intervals,  so  they  were  able  to  track  trends.  They  reviewed  30  to  60 
charts  in  each  round.  The  measures  these  sites  monitored  were: 

•  percentage  of  asthma  patients  with  documented  level  of  severity 
(three  sites) 

•  percentage  of  asthma  patients  with  spirometry  in  last  12  months 
(three  sites) 

•  percentage  of  asthma  patients  with  written  action  plan  (three 
sites) 

•  percentage  of  asthma  patients  with  medication  appropriate  to 
level  of  severity  (two  sites) 

•  percentage  of  asthma  patients  attending  patient  education  classes 
(two  sites) 

•  percentage  of  asthma  patients  using  long-term  medications  (one 
site). 

Note  that  this  set  of  measures  that  the  MTFs  actually  monitored 
differed  somewhat  from  the  set  they  originally  had  planned  to  moni¬ 
tor,  which  are  listed  in  Table  4.3.  One  MTF  dropped  its  use  of 
appropriate  diagnosis  of  asthma  and  appropriate  telephone  triage  to 
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emergency  care.  Two  sites  added  attendance  at  patient  education 
classes,  and  one  site  added  use  of  long-term  medications. 

The  sites  identified  several  issues  that  made  chart  reviews  a  poor 
data  source  for  an  ongoing  monitoring  process.  One  issue  they  expe¬ 
rienced  was  the  difficulty  in  locating  patient  charts.  Typically,  charts 
were  found  for  less  than  50  percent  of  the  cases  sampled.  Another 
issue  was  the  difficulty  in  retrieving  data  from  the  charts  because 
documentation  is  not  standardized,  and  relevant  forms  (701-R,  SF- 
600,  patient  action  plan)  may  he  inserted  in  various  places  in  the 
charts.  One  site  partially  circumvented  this  problem  by  reviewing 
charts  on  a  more  frequent,  monthly  basis. 

The  MTF  teams  indicated  that  they  would  welcome  assistance 
from  MEDCOM  in  designing  and  implementing  more  efficient  chart 
reviews.  They  also  indicated  they  would  welcome  the  development  by 
MEDCOM  of  ad  hoc  programs  that  the  sites  could  use  to  retrieve 
trend  data  from  the  ADS  and  CHCS,  which  they  currently  do  not 
have  the  skills  to  do  on  their  own.  Alternatively,  they  suggested  that 
MEDCOM  perform  the  monitoring  function. 

One  site  displayed  the  results  of  its  chart  reviews  separately  for 
each  clinic  and  eventually  planned  to  monitor  the  asthma  indicators 
by  provider.  This  site  reported  the  results  from  its  chart  audits  to  each 
clinic’s  process  improvement  group  for  follow-up  and  appropriate 
action.  A  clinic’s  process  improvement  group  includes  providers  and 
ancillary  staff,  and  it  meets  regularly  to  address  issues  affecting  the 
clinic.  At  the  other  sites,  some  providers  reported  they  had  been 
shown  some  data  on  the  asthma  metrics  during  the  demonstration. 
However,  many  of  the  providers  who  participated  in  our  focus  groups 
reported  that  they  did  not  know  what  the  metrics  were  for  the  asthma 
guideline.  The  sites  had  given  consideration  to  integrating  monitor¬ 
ing  of  the  asthma  metrics  into  their  quality  management  program  or 
peer  review  processes,  but,  as  of  the  time  of  our  last  visit,  none  of 
them  had  taken  actions  toward  this  end. 

The  champions  at  the  sites  raised  the  issue  of  how  to  interpret 
the  results  from  the  chart  reviews.  They  suggested  that  MEDCOM 
set  performance  objectives  for  their  facilities.  This  would  bolster  the 
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role  of  the  champions  hy  making  providers  and  MTF  staff  account¬ 
able  for  meeting  these  ohjectives. 


Lessons  Learned 

MEDCOM  Support 

•  Having  learned  from  an  earlier  implementation  of  a  clinical 
guideline  for  low  hack  pain,  MEDCOM  allocated  adequate  staff 
and  other  resources  to  support  the  implementation  of  the 
asthma  guideline.  As  a  result,  MEDCOM  could  provide  techni¬ 
cal  support  and  toolkits  to  the  MTEs  from  the  beginning  of  the 
demonstration,  and  it  was  responsive  to  their  questions  and 
requests. 

MEDCOM  can  encourage  a  more  proactive  approach  by  MTEs 
by  reviewing  the  action  plans  prior  to  actual  implementation  and 
suggesting  changes  to  strengthen  the  MTE  strategies.  It  also  will  be 
important  to  identify  sources  of  delays  or  inertia  and  work  to  remove 
those  barriers.  Development  of  action  plans  for  implementation  of  a 
clinical  guideline  is  a  critical  step.  Although  the  MTEs  generally 
developed  reasonable  action  plans,  they  could  not  complete  the 
actions  they  had  defined  in  their  implementation  strategies.  In  some 
cases,  factors  out  of  their  control  intervened  (e.g.,  deployments  of 
medical  personnel).  However,  other  delays  in  implementation  could 
have  been  prevented  or  shortened  with  intervention  by  MTE  com¬ 
mand,  the  implementation  teams,  or  clinic  providers. 

•  The  demonstration  confirmed  that  more  work  was  needed  on 
the  MEDCOM  asthma  encounter  documentation  form  to  make 
it  usable  (and  acceptable)  for  providers.  Many  providers  did  not 
like  the  current  form,  stating  that  it  lacked  adequate  room  to 
write  notes  and  was  not  practical  to  use  for  multiple-diagnosis 
cases,  which  reportedly  represent  a  large  share  of  a  primary  care 
provider’s  caseload. 
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The  documentation  forms  should  he  redesigned  to  he  simple 
and  should  leave  ample  room  for  notes.  Also,  the  number  of  forms 
should  he  limited  to  a  strict  minimum.  This  demonstration  suggested 
that  when  MTFs  are  provided  with  a  considerable  amount  of  new 
forms  (four  in  this  demonstration),  they  may  be  likely  to  disregard 
them. 

•  Once  MEDCOM  has  revised  its  guideline  documentation 
forms,  it  may  want  to  consider  making  use  of  the  forms  manda¬ 
tory.  Demonstration  results  for  both  the  asthma  and  low  back 
pain  guidelines  suggest  that  when  form  use  is  voluntary,  many 
MTFs  and  primary  care  providers  will  choose  not  to  use  it,  pre¬ 
ferring  to  develop  and  use  their  own  forms.  Such  inconsistency 
in  documentation  will  make  it  more  difficult  for  MEDCOM  to 
monitor  performance  on  the  guideline  metrics. 

•  A  related  issue  regarding  use  of  standardized  forms  is  the 
approach  taken  by  MEDCOM  to  develop  a  separate  documen¬ 
tation  form  for  each  guideline  it  is  implementing.  With  multiple 
clinical  guidelines  being  implemented,  MTFs  and  primary  care 
providers  will  be  faced  with  several  forms  for  the  various  guide¬ 
line  conditions,  and  their  response  may  be  to  ignore  all  of  them. 
This  issue  could  be  alleviated  by  establishing  an  automated 
medical  record,  which  is  under  development  by  DoD. 

•  MEDCOM  should  provide  continuing  leadership  in  establish¬ 
ing  mechanisms  to  maintain  effective  communications  with  the 
MTFs  on  guideline  implementation  activities.  As  we  learned  in 
both  the  asthma  and  low  back  pain  guideline  demonstrations, 
daily  demands  on  the  time  of  MTF  staff  impede  their  ability 
and  willingness  to  take  initiative  to  communicate  with  other 
MTFs.  Thus,  any  mechanisms  established  for  cross-MTF  com¬ 
munications  should  be  easy  to  access  and  use,  to  avoid  barriers 
that  might  further  constrain  communication  activities. 
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Support  at  the  MTF 

•  The  command  teams  of  the  participating  MTFs  need  to  provide 
visible  and  active  support  for  use  of  the  practice  guideline  to 
make  clear  that  leadership  has  placed  a  high  priority  on  achiev¬ 
ing  the  best  practices  delineated  in  the  guideline.  This  involves 
making  clear  statements  of  support,  demanding  accountability 
for  making  change  happen,  and  providing  appropriate  staff 
resources  for  a  reasonable  implementation  period.  Such  support 
will  increase  motivation  for  rank-and-file  providers,  who  should 
be  more  likely  to  engage  in  use  of  the  guideline  practices  when 
they  know  they  will  be  evaluated  on  how  well  they  do  so. 

•  Regular  meetings  with  the  MTF  command  need  to  be  scheduled 
to  provide  updates  on  progress  in  carrying  out  the  implementa¬ 
tion  action  plan,  to  review  results  from  the  monitoring  of  met¬ 
rics,  and  to  resolve  issues  that  arise  as  new  practices  are  being 
introduced. 

•  Additional  consideration  should  be  given  to  the  role  and  compo¬ 
sition  of  the  MTF  implementation  team.  With  the  exception  of 
those  involved  in  monitoring  the  guideline  metrics,  implementa¬ 
tion  team  members  generally  were  not  actively  engaged  in  car¬ 
rying  out  the  action  plans.  Despite  limited  roles,  their  presence 
on  the  team  contributed  to  building  ownership  for  the  guideline 
and  identifying  feasible  approaches  to  improving  practices. 
Increased  involvement  should  lead  to  even  stronger  ownership. 
Mechanisms  also  need  to  be  put  in  place  to  facilitate  regular 
communications  among  team  members  so  they  can  troubleshoot 
issues  and  assess  progress  in  carrying  out  their  action  plan.  This 
mechanism  could  be  regular  team  meetings  or  other  alternatives, 
such  as  e-mail,  that  might  work  as  well  or  better,  depending  on 
the  team  members’  preferences. 


CHAPTER  FIVE 


Effects  of  Guideline  Implementation 


In  this  chapter,  we  examine  the  extent  to  which  introduction  of  the 
asthma  guideline  at  the  four  demonstration  MTFs  changed  clinical 
practices  at  those  facilities.  We  use  information  from  the  process 
evaluation  to  assess  how  well  providers  at  the  demonstration  sites 
appeared  to  understand  and  accept  the  guideline.  We  also  summarize 
reports  from  the  sites  on  their  monitoring  of  the  effects  of  implemen¬ 
tation  activities  on  clinical  practices.  We  then  report  results  from  our 
analysis  of  encounter  data  on  trends  in  clinical  practices  for  the  indi¬ 
cators  of  asthma  care,  comparing  the  demonstration  sites  to  control 
sites  that  had  not  used  the  guideline.  Finally,  we  compare  MTF  costs 
for  health-care  service  to  enrolled  asthma  patients  for  the  two  study 
years  and  assess  any  differences  between  the  demonstration  and  con¬ 
trol  MTFs. 


Provider  Knowledge  and  Acceptance  of  the  Guideline 

There  was  positive  to  neutral  acceptance  of  the  asthma  practice 
guideline  hy  participants  at  the  demonstration  MTFs.  Providers  at 
two  MTFs  indicated,  however,  that  it  would  not  change  their  prac¬ 
tices.  One  of  these  MTFs  was  already  using  an  asthma  guideline 
similar  to  the  DoD/VA  guideline,  and  participants  there  saw  no  rea¬ 
son  to  modify  their  practices.  At  the  other  MTF,  providers  believed 
their  practices  were  already  consistent  with  the  guideline. 
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These  reactions  of  the  site  implementation  teams  were  consis¬ 
tent  with  attitudes  expressed  hy  primary  care  providers  interviewed 
during  the  RAND  site  visits,  which  ranged  from  strong  support  to 
passive  acceptance.  Some  providers  said  they  were  already  practicing 
according  to  the  guideline’s  standard  of  care,  and  some  saw  guidelines 
in  general  as  a  threat  to  their  autonomy  and  not  especially  useful. 
Some  positive  comments  also  were  heard  from  providers: 

•  “This  guideline  is  long  overdue.” 

•  “Its  utilization  will  lead  to  good  patient  outcomes.” 

•  “We  are  pleasantly  surprised  hy  the  quality  of  the  guideline.” 

•  “Its  focus  on  severity  is  helpful.” 

•  “It  makes  things  concrete,  provides  uniformity  of  care,  and 
serves  as  a  reminder.” 

At  one  site,  the  guideline  conflicted  with  local  practice.  Provid¬ 
ers  at  that  site  did  not  prescribe  peak  flow  meters  to  patients  diag¬ 
nosed  with  mild  persistent  asthma.  They  also  pointed  out  that  there 
was  no  medical  evidence  to  support  the  guideline’s  requirement  of  an 
annual  spirometry  reading. ' 

At  the  end  of  the  demonstration,  providers  participating  in  our 
focus  groups  were  asked  their  views  about  clinical  practice  guidelines 
in  general  and  the  asthma  guideline  in  particular.  Most  of  them 
reported  that  the  asthma  guideline  had  taught  them  aspects  of  care 
they  did  not  know  before  and  had  helped  them  provide  better  care  to 
their  asthma  patients.  Most  also  felt  that  the  asthma  guideline  had 
not  reduced  their  flexibility  to  treat  asthma  patients,  but  they  did 
report  that  using  the  guideline  increased  the  time  they  spent  with 
asthma  patients. 


*  This  site’s  champion,  who  also  had  participated  in  the  development  of  the  guideline, 
agreed  that  no  medical  evidence  supported  this  requirement  for  annual  spirometry  but 
reported  that  it  had  been  included  in  the  belief  that  it  would  not  get  done  if  it  were  not 
required  regularly.  Regular  spirometry  reading  was  also  a  perceived  aid  in  assessing  the  effec¬ 
tiveness  of  treatment. 
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These  results  suggest  that  providers  were  generally  positive  about 
the  asthma  guideline,  hut  that  taking  all  the  steps  the  guideline  speci¬ 
fies  might  harm  their  daily  productivity.  In  addition,  a  significant 
minority  of  providers  reported  that  the  guideline  did  not  help  them. 


Reported  Changes  in  Clinical  Practices 

To  learn  about  changes  in  clinical  practices  made  by  the  demonstra¬ 
tion  sites,  we  asked  the  implementation  team  and  providers  in  our 
focus  groups  at  each  site  to  identify  which  practice  changes  they  made 
and  whether  they  had  evidence  that  documented  such  changes.  We 
also  reviewed  progress  reports  from  these  sites  and  data  they  had 
developed  on  trends  in  the  asthma  metrics.  Overall,  we  found  evi¬ 
dence  of  some  changes  to  make  clinic  practices  more  consistent  with 
the  guideline’s  recommendations,  but  these  changes  were  neither 
strong  nor  widespread  among  the  participating  providers  and  clinics. 
Some  examples  of  the  reported  changes  are  summarized  below. 

Changes  in  Referral  Patterns 

Providers  participating  in  our  final  site  visit  were  asked  if  the  asthma 
guideline  led  them  to  increase  or  decrease  referrals  to  a  number  of 
specified  services.  Providers  at  all  four  sites  reported  that  they  had 
increased  referrals  of  patients  to  education  programs  and  to  spirome¬ 
try.  At  two  sites,  they  reported  they  had  increased  referrals  to  case 
management.  Providers  also  reported  that  they  increased  referrals  to 
allergy,  smoking  cessation,  or  respiratory  therapy.  For  all  of  these 
increases  in  referrals,  however,  providers  reported  that  their  changes 
in  referrals  were  not  extensive.  Finally,  providers  reported  no  changes 
to  their  pattern  of  referrals  for  X  rays,  tests  for  airway  function,  pul¬ 
monology,  or  the  Medical  Evaluation  Board. 

These  reports  represent  the  perceptions  of  the  providers  that 
participated  in  our  focus  groups,  which  may  not  be  representative  of 
all  providers  at  the  demonstration  sites,  who  may  have  had  less  expo¬ 
sure  to  use  of  the  practice  guideline.  These  perceptions  also  may  dif- 
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fer  from  objective  data  on  changes  in  clinical  practices,  as  measured 
with  encounter  or  chart  data. 

Changes  in  Asthma  Indicators  Monitored  by  the  Sites 

Three  of  the  four  demonstration  sites  performed  reviews  of  randomly 
drawn  samples  of  asthma  patient  charts  at  two  points  in  time — early 
in  the  demonstration  and  again  a  few  months  later.  The  reviews  were 
done  to  monitor  documentation  of  compliance  with  the  clinical  prac¬ 
tices  recommended  by  the  asthma  guideline  and  to  assess  the  extent 
to  which  practices  had  changed  over  time. 

We  provide  here  a  qualitative  summary  of  the  results  of  the 
MTFs’  chart  data  analyses,  which  gives  some  sense  of  how  the  MTFs 
perceived  they  were  progressing.  The  validity  of  the  data  is  uncertain 
because  they  worked  with  small  samples  of  cases  and  they  did  not 
have  a  standard  chart  abstraction  method  to  use.  In  particular,  the 
teams  performing  the  reviews  could  find  fewer  than  half  of  the 
records  drawn  for  the  sample,  which  increases  the  probability  of  bias. 

The  three  sites  that  monitored  the  asthma  metrics  with  chart 
data  reported  an  increase  in  the  documentation  of  asthma  severity 
level,  as  well  as  an  increase  in  the  percentage  of  patients  for  whom  use 
of  spirometry  and  written  patient  action  plans  were  documented. 
However,  they  found  that  the  level  of  compliance  with  documenta¬ 
tion  remained  low  (according  to  their  chart  data,  below  40  percent) . 
Two  sites  reported  no  change  in  attendance  at  patient  education.  One 
site  reported  an  increase  in  the  percentage  of  patients  for  whom  use  of 
long-term  controllers  was  documented.  Another  reported  an  increase 
in  the  percentage  of  patients  for  whom  treatment  provided  was 
appropriate  for  the  level  of  asthma  severity  recorded. 

Changes  in  Asthma  Medication  Prescriptions 

Providers  participating  in  our  focus  groups  were  asked  to  indicate 
whether  they  changed  the  way  they  prescribed  specified  pharmaceuti¬ 
cals  in  the  management  of  asthma  patients.  Some  of  them  (but  not  a 
majority)  reported  that  they  had  increased  use  of  long-term  controller 
medications,  inhaled  steroids,  and  long-acting  Beta2  agonists.  Again, 
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these  reported  patterns  of  medication  use  for  asthma  patients  reflect 
provider  perceptions  and  may  differ  from  actual  patterns  of  use. 


Analysis  of  Effects  on  Service  Delivery 

For  the  analysis  of  trends  in  service  delivery  for  asthma  patients,  we 
compared  the  practices  of  the  four  demonstration  sites  before  guide¬ 
line  implementation  to  the  practices  after  implementation  and  also  to 
the  practices  of  six  control  sites  that  were  not  part  of  the  demonstra¬ 
tion.  Refer  to  Chapter  Two  and  Appendix  B  for  details  on  the  meth¬ 
ods  used  for  this  analysis. 

The  Study  Sample 

As  described  in  Chapter  Two,  the  patient  sample  used  for  the  analysis 
of  guideline  effects  on  asthma  care  at  the  demonstration  MTFs  was 
the  MTF  enrollees — i.e.,  TRICARE  Prime  enrollees  who  had  a  PCM 
based  at  one  of  these  MTFs.  This  group  is  a  subset  of  the  Army 
asthma  population  we  refer  to  as  MTF  users — i.e.,  all  asthma  patients 
who  received  care  at  one  of  the  demonstration  or  control  MTFs  dur¬ 
ing  the  period  of  the  study.  MTF  enrollees  received  most,  if  not  all, 
of  their  care  from  the  MTFs  at  which  they  were  enrolled,  but  it  does 
not  exclude  use  of  services  from  other  providers. 

The  study  sample  by  MTF  is  shown  in  Table  5.1.  Patients 
enrolled  at  the  demonstration  MTFs  constituted  35.7  percent  of  the 
total  sample  in  the  first  study  year  and  37.9  percent  in  the  second 
year. 

Measures  and  Methods 

As  described  in  Chapter  Two  and  Appendix  B,  six  indicators  were 
selected  for  analysis  in  the  evaluation  (see  Table  2.4).  In  Chapter 
Three,  we  presented  the  baseline  performance  on  these  indicators  for 
all  the  demonstration  and  control  MTFs.  In  this  chapter,  we  examine 
the  extent  to  which  the  indicators  changed  for  the  demonstration 
MTFs  during  the  study  period  as  they  implemented  new  practices  for 
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Table  5.1 

Number  and  Percentage  of  Asthma  Patients  Enrolled  in 
TRICARE  Prime  at  a  Demonstration  or  Control  MTF  (MTF 
Enrollees),  by  MTF 


Study  Year  One 

Study  Year  Two 

Site 

Number  of 
Patients 

Percentage 

Number  of 
Patients 

Percentage 

All  sites 

12,970 

100.0 

11,320 

100.0 

Demo  sites 

Demo  1 

1,176 

9.1 

1,059 

9.4 

Demo  2 

1,159 

8.9 

1,043 

9.2 

Demo  3 

1,715 

13.2 

1,628 

14.4 

Demo  4 

581 

4.5 

559 

4.9 

All  demos 

4,631 

35.7 

4,289 

37.9 

Control  sites 

Control  1 

1,635 

12.6 

1,280 

11.3 

Control  2 

785 

6.1 

659 

5.8 

Control  3 

736 

5.7 

595 

5.3 

Control  4 

2,621 

20.2 

2,050 

18.1 

Control  5 

1,747 

13.5 

1,558 

13.8 

Control  6 

815 

6.3 

889 

7.9 

All  controls 

8,339 

64.3 

7,031 

62.1 

NOTE:  MTF  enro//ees signifies  beneficiaries  who  were  enrolled  in 
TRICARE  Prime  at  this  MTF  for  all  of  their  inpatient  and 
outpatient  encounters  in  the  SIDR  and  SADR  data. 


the  asthma  guideline.  Our  analysis  tests  the  hypotheses  listed  in  Table 

2.1. 

All  of  the  demonstration  MTFs  decided  to  start  implementing 
the  asthma  guideline  in  one  or  two  of  their  primary  care  clinics, 
rather  than  attempting  to  introduce  practice  changes  across  all  clinics 
serving  asthma  patients.  Each  chose  the  clinics  that  saw  the  largest 
share  of  their  asthma  patients.  Because  any  changes  in  the  asthma 
indicators  in  our  analysis  would  have  occurred  only  for  patients 
enrolled  with  PCMs  in  the  participating  clinics,  we  separated  data  for 
patients  enrolled  at  those  clinics  from  that  for  the  rest  of  the  asthma 
patients  enrolled  at  the  demonstration  MTFs.  We  refer  to  these  clin¬ 
ics  as  the  “target  clinics”  in  the  presentation  of  our  results. 

For  each  indicator,  we  present  a  figure  displaying  trends  in  the 
average  values  for  the  demonstration  sites  and  control  sites.  For  the 
three  medication  measures,  annual  values  are  reported  for  each  of  the 
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two  Study  years.  For  the  three  measures  of  service-use  rates,  we  report 
quarterly  trends  in  annualized  rates  for  the  eight  quarters  included  in 
the  study  period.  Tables  with  data  that  support  these  figures,  includ¬ 
ing  individual  rates  for  each  MTF,  can  he  found  in  Appendix  E. 

To  test  the  significance  of  any  observed  trends,  we  estimated  a 
multivariate  regression  model  for  each  indicator  for  which  the  patient 
was  the  unit  of  analysis.  The  dependent  variable  of  each  model  was 
the  indicator  being  tested,  and  predictor  variables  were  demonstration 
versus  control,  individual  facility,  and  calendar  quarter.  We  con¬ 
trolled  for  patient  characteristics  of  military  status,  gender,  and  age 
categories  in  these  models.  We  could  not  control  for  differences  in 
asthma  severity  level  because  administrative  data  do  not  contain  this 
information.  Interaction  terms  were  used  to  test  differences  in  rates 
during  the  demonstration  period  relative  to  the  two  baseline  quarters. 

Use  of  Long-Term  Controller  Medications 

The  asthma  guideline  recommends  use  of  long-term  controller  medi¬ 
cations  for  patients  with  asthma  severity  greater  than  mild  intermit¬ 
tent.  We  hypothesized  that  introducing  more  active  management  of 
asthma  patients  would  increase  the  percentage  of  patients  with  pre¬ 
scriptions  for  long-term  controllers,  to  reduce  the  frequency  of 
asthma  exacerbations. 

As  shown  in  Figure  5.1,  the  target  demonstration  group  had  the 
highest  percentage  of  asthma  patients  with  prescriptions  for  long¬ 
term  controller  medications  in  both  study  years  one  and  two.  The 
percentages  declined  from  the  first  to  second  study  year  for  all  groups. 
Thus,  we  found  no  evidence  of  the  hypothesized  increase  in  use  rates 
of  long-term  controllers  under  the  guideline. 

We  hypothesized  that  introduction  of  more  active  management 
of  asthma  patients  would  increase  use  rates  of  complementary  medi¬ 
cations  to  support  improved  control  of  exacerbations.  The  percentage 
of  asthma  patients  who  were  prescribed  complementary  medications 
was  substantially  larger  in  the  target  demonstration  group  than  in 
either  the  other  demonstration  or  control  groups,  as  shown  in  Figure 
5.2.  However,  we  found  no  evidence  of  the  hypothesized  increase  in 
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Figure  5.1 

Prescription  of  Long-Term  Controllers,  for  Target  Demonstration,  Other 
Demonstration,  and  Control  Sites,  by  Year 
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use  of  complementary  medications  under  the  guideline.  The  percent¬ 
ages  of  patients  using  these  medications  declined  between  study  years 
one  and  two  for  all  three  groups. 

Use  of  Short-Acting  Rescue  Medications  for  Asthma  Exacerbations 

With  its  emphasis  on  reducing  frequency  and  severity  of  exacerba¬ 
tions,  the  asthma  practice  guideline  recommends  that  asthma  patients 
have  available  short-acting  rescue  medications  for  early  use  at  times 
they  begin  having  symptoms  of  an  exacerbation.  Thus,  we  hypothe¬ 
sized  that  introduction  of  more  active  management  of  asthma 
patients  would  increase  the  percentage  of  patients  with  prescriptions 
for  rescuer  medications. 

Relatively  large  percentages  of  asthma  patients  had  prescriptions 
for  rescue  medications  in  the  first  study  year,  as  shown  in  Figure  5.3. 
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Figure  5.2 

Prescription  of  Complementary  Medications,  for  Target  Demonstration, 
Other  Demonstration,  and  Control  Sites,  by  Year 


RAND  MG319-5.2 


Again,  the  target  demonstration  group  had  the  largest  average  per¬ 
centage  (77.0  percent),  followed  hy  the  control  sites  (65.3  percent) 
and  the  other  demonstration  group  (59.2  percent).  The  average  per¬ 
centage  for  the  target  and  other  demonstration  groups  declined 
slightly  in  study  year  two,  while  those  for  the  control  group  remained 
essentially  the  same.  Thus,  we  found  no  evidence  of  the  hypothesized 
increase  in  use  of  rescue  medications  under  the  guideline. 

Use  of  Outpatient  Services 

In  Figure  5.4,  we  present  annualized  rates  of  asthma-related  outpa¬ 
tient  visits  per  1 00  asthma  patients  for  each  of  the  eight  quarters  in 
the  study  period,  where  a  visit  was  determined  to  he  asthma-related  if 
the  asthma  diagnostic  code  was  reported  on  the  encounter  record.  To 
the  extent  that  introduction  of  the  practice  guideline  affected  these 
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Figure  5.3 

Prescription  of  Short-Acting  Rescue  Medications,  for  Target  Demonstration, 
Other  Demonstration,  and  Control  Sites,  by  Year 
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use  rates,  we  should  see  a  change  in  the  trend  starting  in  the  fifth 
quarter,  which  is  the  first  quarter  after  the  MTFs  began  to  work  with 
the  guideline  in  the  target  clinics. 

We  observe  what  appears  to  be  a  seasonal  pattern  of  variation  in 
outpatient  use  rates  for  patients  in  all  three  groups.  The  rates  are 
highest  in  quarters  one  and  five  (January  through  March  of  each  year) 
and  lowest  in  quarters  three  and  seven  (July  through  September  of 
each  year).  If  introduction  of  the  guideline  affected  outpatient  use 
rates,  we  should  observe  a  decline  in  the  use  rates  for  the  target  dem¬ 
onstration  relative  to  the  other  two  groups,  independent  of  the 
seasonal  pattern.  Such  a  decline  is  not  found,  indicating  that  the 
guideline  had  no  effect  on  overall  use  of  outpatient  services  in  the 
demonstration  sites. 
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Figure  5.4 

Trends  in  Asthma-Related  Outpatient  Visit  Rates,  by  Target  Demonstration, 
Other  Demonstration,  and  Control  Sites 


NOTE:  Annualized  outpatient  visit  rates  are  calculated  as  four  times  the  number  of 
visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant  study  year  and 
applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per  100  patients. 

RAND  MC319-S.4 


Use  of  Emergency  Room  Services 

According  to  the  practice  guideline,  effective  management  of  asthma 
should  lead  to  less  frequent  exacerbations,  which  in  turn  should 
reduce  rates  of  emergency  room  use  hy  asthma  patients  suffering  an 
exacerbation.  Thus,  we  hypothesized  that  introduction  of  new  prac¬ 
tices  specified  in  the  asthma  guideline  would  contribute  to  a  reduc¬ 
tion  in  asthma-related  emergency  room  use  rates  by  asthma  patients 
served  by  the  demonstration  MTFs. 

We  measured  emergency  room  use  as  the  number  of  asthma- 
related  emergency  room  visits  per  100  asthma  patients.  Annualized 
use  rates  were  calculated  for  each  quarter  of  the  two  study  years.  Con¬ 
trol  site  six  was  excluded  from  this  analysis  because  it  did  not  have  an 
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emergency  room,  so  its  patients  had  to  use  other  facilities  for  which 
our  data  was  incomplete.  ^ 

As  shown  in  Figure  5.5,  the  three  groups  of  target  demonstra¬ 
tion,  other  demonstration,  and  control  sites  had  similar  levels  of 
asthma-related  emergency  room  visit  rates,  with  the  control  sites’ 
rates  slightly  higher  than  those  for  the  two  demonstration  groups. 
Visit  rates  for  patients  served  hy  the  target  demonstration  clinics 
declined  in  the  second  study  year,  as  hypothesized,  hut  so  did  rates 


Figure  5.5 

Trends  in  Asthma-Related  Emergency  Room  Visit  Rates,  by  Target 
Demonstration,  Other  Demonstration,  and  Control  Sites 


NOTES:  Excludes  control  site  six,  which  does  not  have  an  emergency  room.  Annualized 
emergency  room  visit  rates  are  calculated  as  four  times  the  number  of  visits  in  a 
quarter  divided  by  the  number  of  patients  in  the  relevant  study  year  and  applying  100 
as  an  adjustment  factor  to  standardize  to  a  rate  per  100  patients. 

RAND  MC319-5.S 


^  We  repeated  this  analysis  with  all  the  control  sites  and  obtained  the  same  results  we  found 
for  the  analysis  reported  here  that  excluded  control  site  six. 
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for  the  control  sites.  Therefore,  we  cannot  conclude  that  the  decline 
for  the  target  demonstration  clinics  is  attributable  to  introduction  of 
the  practice  guideline.  The  absence  of  effects  was  confirmed  by 
regression  modeling,  which  found  no  significant  difference  between 
the  groups  in  the  reduction  of  emergency  room  visit  rates  for  the  sec¬ 
ond  study  year. 

Use  of  Hospital  Inpatient  Services 

Similar  to  the  hypothesis  for  emergency  room  visit  rates,  we  hypothe¬ 
sized  that  use  of  the  asthma  guideline  would  reduce  rates  of  asthma- 
related  hospitalizations  for  patients  at  the  demonstration  sites  during 
the  second  study  year.  This  effect  would  derive  from  improved  man¬ 
agement  of  care  for  asthma  patients  by  the  participating  MTFs.  In 
calculating  hospitalization  rates,  we  included  in  the  counts  fall  admis¬ 
sions  to  either  MTFs  and  network  provider  hospitals  that  had  a 
diagnosis  code  for  asthma  in  any  diagnosis  field.  Annualized  hospi¬ 
talization  rates  per  1 00  asthma  patients,  calculated  for  each  quarter  of 
the  two  study  years,  are  shown  in  Figure  5.6.  These  rates  offer  no 
evidence  for  the  hypothesized  reduction  in  hospitalization  rates  for 
the  demonstration  sites.  Hospitalization  rates  remained  steady  for  the 
target  demonstration  group  as  well  as  for  the  control  sites.  Only  the 
rates  for  the  other  demonstration  group  declined  in  the  second  study 
year,  which  we  interpret  as  an  unrelated  trend  that  cannot  be  attrib¬ 
uted  to  use  of  the  asthma  guideline  because  the  sites  had  undertaken 
interventions  only  in  the  target  clinics. 

Asthma-related  inpatient  stays  represented  only  45  percent  of 
total  inpatient  stays  for  the  target  demonstrations  and  were  33  per¬ 
cent  of  total  stays  for  the  other  demonstration  group  and  control 
group.  We  also  tested  for  trends  in  total  hospitalization  rates  for 
patients  in  the  three  groups,  to  examine  whether  asthma  management 
might  be  having  a  measurable  impact  on  other  reasons  for  hospitaliza¬ 
tion,  and  we  found  no  significant  change  in  these  rates  between  the 
two  study  years. 
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Figure  5.6 

Trends  in  Asthma-Related  Hospital  Inpatient  Stays,  by  Target 
Demonstration,  Other  Demonstration,  and  Control  Sites 


Quarter 

NOTE:  Annualized  hospitalization  rates  are  calculated  as  four  times  the  number  of  in¬ 
patient  stays  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant  study  year 
and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per  100  patients. 
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Estimated  Costs  of  Care 

We  began  the  cost  analysis  by  looking  at  the  total  costs  of  care  for  the 
MTF  users  (all  asthma  patients  served  by  the  ten  MTFs  that  were 
demonstration  or  control  sites  for  this  evaluation).  We  looked  at  both 
the  costs  experienced  by  the  MTFs  for  asthma  care  and  those  experi¬ 
enced  by  the  patients  themselves.^  Costs  were  broken  down  by  two 
subgroups  of  the  MTF  users — MTF  enrollees  (those  enrolled  in 
TRICARE  Prime  at  the  MTF)  and  nonenrollees. 


^  As  discussed  in  Chapter  Two,  some  uncertainty  exists  about  the  quality  of  MEPRS  cost 
data,  and  the  data  quality  varies  across  MTFs. 
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From  the  MTF  perspective,  the  lion’s  share  of  costs  for  asthma 
patients  are  for  nonenrollees,  as  shown  in  Figure  5.7.  Less  than  half  of 
the  total  costs  of  care  for  asthma  patients  for  both  demonstration  and 
control  MTFs  were  spent  on  enrollees.  The  shares  of  total  costs  for 
enrollees  declined  to  about  40  percent  for  both  MTF  groups  in  the 
second  year.  The  demonstration  MTFs  incurred  a  somewhat  smaller 
share  of  their  total  costs  for  MTF  enrollees  than  did  the  control  sites, 
and  this  difference  persisted  in  both  study  years.  As  discussed  above, 
it  is  more  difficult  for  primary  care  providers  to  manage  asthma  care 
for  nonenrollees  because  these  patients  use  MTF  services  episodically 
and  many  lack  a  regular  primary  care  provider  at  the  MTF. 

Figure  5.8  shows  estimated  average  MTF  costs  from  the  patient 
perspective,  including  total  costs  per  asthma  patient  and  components 

Figure  5.7 

Composition  of  MTF  Total  Costs  for  Asthma  Patients  Who  Were  MTR 
Enrollees  and  Nonenrolled  Users  for  Demonstration  and  Control  Sites,  by 
Study  Year  (FY  1999  $) 
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Figure  5.8 

Estimated  Total  MTF  Cost  per  Asthma  Patient  for  MTF  Enrollees  and  Other 
Users  of  the  Demonstration  and  Control  Sites,  by  Study  Year  (FY  1999  $) 
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of  costs  for  outpatient  and  inpatient  services.  Asthma  patients 
enrolled  at  MTFs  had  substantially  lower  total  MTF  costs  per 
patient,  on  average,  than  those  using  the  MTFs  on  a  more  episodic 
basis,  a  difference  that  was  found  for  both  study  years.  This  difference 
in  costs  is  attributable  to  much  higher  MTF  inpatient  service  costs 
per  capita  for  nonenrollees. 

Several  possible  explanations  might  contribute  to  the  lower  costs 
experienced  by  MTF  enrollees.  The  enrollees  might  have  received 
more  regular  care  from  the  MTFs  that  prevented  conditions  or  com¬ 
plications  that  would  lead  to  a  need  for  hospital  stays  or  that  reduced 
the  severity  of  treatment  needed  during  an  inpatient  stay.  An  alterna¬ 
tive  explanation  might  be  that  nonenrollees  were  more  likely  to  seek 
care  only  when  they  experienced  a  health  problem,  at  which  time 
they  would  make  heavier  use  of  specialty  care,  hospitalization,  and 
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Other  more  costly  services  when  they  entered  the  system.  Enrollees 
would  he  receiving  more  regular  care  in  primary  care  clinics,  which 
have  lower  unit  costs  than  specialty  care.  One  might  also  speculate 
that  enrollees  obtain  relatively  more  inpatient  care  from  community 
providers  than  the  nonenrollees  do,  hut  this  explanation  is  counterin¬ 
tuitive  because  enrollees  are  expected  to  obtain  care  at  the  MTF 
where  they  are  enrolled  except  when  the  MTF  does  not  offer  the 
services  needed. 

Overall  Costs  of  MTF  Services 

In  this  part  of  the  cost  analysis,  we  examine  overall  average  costs  for 
asthma  patients  at  the  demonstration  and  control  MTFs,  comparing 
costs  for  study  years  one  and  two.  We  consider  separately  average 
costs  for  MTF  enrollees  and  for  those  who  were  not  enrolled  but  used 
the  MTF  services.  As  described  earlier  in  the  report,  our  cost  study 
had  two  objectives:  to  gain  an  understanding  of  the  costs  that  MTFs 
incur  for  health-care  services  for  asthma  patients  and  to  evaluate 
whether  introduction  of  the  asthma  practice  guideline  had  observable 
effects  on  those  costs  for  MTFs  participating  in  the  demonstration. 

Costs  for  MTF  enrollees.  The  aggregate  and  per-patient  MTF 
costs  for  asthma  patients  enrolled  at  the  MTFs  are  presented  in  Table 
5.2.  The  demonstration  MTFs  incurred  an  estimated  $1.7  million  in 
costs  for  services  to  enrolled  asthma  patients  in  the  first  study  year, 
and  their  costs  decreased  to  $1.5  million  in  the  second  study  year. 
This  decrease  was  the  result  of  both  a  decrease  in  the  number  of 
patients  (from  4,631  to  4,289  patients)  and  a  slight  decline  in  per- 
patient  costs  (from  $374  to  $348  per  patient).  These  cost  estimates 
were  smaller  than  those  for  the  control  MTFs,  which  incurred  an 
estimated  $2.6  million  in  the  first  year  and  $2.1  million  in  the  second 
year.  The  larger  costs  for  the  control  MTFs  reflected  their  larger 
number  of  asthma  patients — 75  to  80  percent  more  asthma  patients 
were  enrolled  at  the  control  MTFs  than  at  the  demonstration  MTFs. 
At  the  same  time,  the  control  MTFs  had  much  lower  total  costs  per 
patient  ($310  and  $296  per  patient  in  years  one  and  two,  respec¬ 
tively)  than  the  demonstration  MTFs. 
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Table  5.2 

Estimated  MTF  Costs  for  Asthma  Patients  Who  Were  MTF  Enrollees 
at  the  Demonstration  and  Control  MTFs,  Study  Years  One  and  Two 


All  Sites 

Demonstration 

Sites 

Control  Sites 

study  year  one 

Number  of  patients 

12,970 

4,631 

8,339 

Aggregate  costs 

Total 

$4,312,885 

$1,730,737 

$2,582,148 

Outpatient 

2,273,675 

929,529 

1,344,146 

Inpatient 

2,039,210 

801,208 

1,238,002 

Costs  per  patient 

Total 

332 

374 

309 

Outpatient 

175 

201 

161 

Inpatient 

157 

173 

148 

Study  year  two 

Number  of  patients 

11,320 

4,289 

7,031 

Aggregate  costs 

Total 

$3,573,992 

$1,493,795 

$2,080,197 

Outpatient 

1,882,271 

818,878 

1,063,393 

Inpatient 

1,691,721 

674,917 

1,016,804 

Costs  per  patient 

Total 

315 

348 

296 

Outpatient 

166 

191 

151 

Inpatient 

149 

157 

145 

NOTE:  All  costs  are  in  1999  dollars  that  were  estimated  by  adjusting  1998 
unit  cost  estimates  by  an  inflation  factor  of  1.4  percent. 


In  the  first  study  year,  outpatient  services  for  asthma  patients 
represented  an  estimated  53.7  percent  of  the  total  estimated  costs  for 
the  demonstration  MTFs,  compared  to  an  estimated  52. 1  percent  for 
the  control  MTFs.  The  second  study  year  mix  of  costs  for  inpatient 
and  outpatient  services  were  similar  to  the  first  year  for  both  the 
demonstration  and  control  MTFs. 

With  the  introduction  of  new  care  management  practices  to 
prevent  asthma  exacerbations,  we  would  expect  outpatient  use  rates 
and  related  costs  for  the  demonstration  MTFs  to  increase  and  inpa¬ 
tient  care  and  costs  to  decrease  as  this  care  helped  reduce  the  fre¬ 
quency  of  exacerbations.  As  shown  in  Table  5.2,  the  total  average  cost 
per  patient  for  asthma  patient  enrollees  decreased  for  both  the  dem¬ 
onstration  and  control  MTFs  (by  6.8  percent  and  4.5  percent,  respec¬ 
tively)  from  baseline  to  study  year  two.  For  demonstration  MTF 
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enrollees,  the  average  outpatient  cost  per  patient  decreased  by  4.8 
percent  (from  $20 1  to  $191)  and  the  average  inpatient  cost  decreased 
by  9.0  percent  (from  $173  to  $157).  At  the  same  time,  the  average 
outpatient  cost  per  patient  for  enrollees  at  the  control  MTFs 
decreased  by  6.3  percent  and  average  inpatient  cost  decreased  by  2.6 
percent. 

The  change  in  per-capita  cost  for  the  demonstration  MTFs  is 
assessed  relative  to  the  change  for  the  control  MTFs,  which  represents 
the  temporal  trend  of  how  costs  might  have  changed  at  the  demon¬ 
stration  MTFs  in  the  absence  of  new  practices  under  the  asthma 
guideline.  The  demonstration  MTFs  experienced  a  net  reduction  in 
per-capita  costs  relative  to  the  change  in  costs  for  the  control  sites. 
The  use  rates  for  outpatient  visits  and  inpatient  stays  did  not  change 
between  study  year  one  and  two,  however,  as  we  reported  above  in 
Figures  5.4  and  5.6.  These  results  suggest  that  there  may  have  been  a 
change  in  mix  of  outpatient  visits  or  inpatient  stays  toward  less  costly 
services. 

Costs  for  nonenrollees.  The  costs  of  care  for  nonenrollees  using 
the  demonstration  MTFs  are  presented  in  Table  5.3.  Total  costs  for 
the  demonstration  MTFs  were  an  estimated  $2.0  million  in  the  first 
study  year,  increasing  to  $2.6  million  in  the  second  year.  The  total 
costs  for  the  control  MTFs  increased  slightly  from  $2.5  million  to 
$2.7  million  between  the  two  years. 

Unlike  the  costs  for  MTF  enrollees,  costs  for  inpatient  services 
represented  more  than  half  of  total  costs  of  care  for  nonenrollees  for 
both  the  demonstration  and  control  MTFs.  Growth  in  the  number  of 
patients  using  MTF  services  contributed  to  increased  aggregate  costs 
for  both  groups. 

Total  per-capita  costs  increased  in  study  year  two  for  nonenrol¬ 
lees  at  the  demonstration  sites  but  declined  for  those  at  the  control 
sites.  Per-capita  outpatient  care  costs  at  the  demonstration  MTFs 
were  virtually  the  same  for  both  study  years  ($217  and  $219  per 
patient),  while  these  costs  increased  slightly  for  the  control  MTFs.  It 
was  the  inpatient  costs  that  accounted  for  the  increase  in  per-capita 
costs  at  the  demonstration  MTFs,  whereas  inpatient  costs  decreased 
slightly  at  the  control  MTFs. 
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Table  5.3 

Estimated  MTF  Costs  for  Nonenrollee  Asthma  Patients  Using  Care  at 
the  Demonstration  and  Control  MTFs,  Study  Years  One  and  Two 


All  Sites 

Demonstration 

Sites 

Control  Sites 

study  year  one 

Number  of  patients 

10,109 

3,808 

6,301 

Aggregate  costs 

Total 

$4,453,834 

$1,990,383 

$2,463,451 

Outpatient 

1,836,008 

826,121 

1,009,887 

Inpatient 

2,617,826 

1,164,262 

1,453,564 

Costs  per  patient 

Total 

441 

523 

391 

Outpatient 

182 

217 

160 

Inpatient 

259 

306 

231 

Study  year  two 

Number  of  patients 

12,060 

4,768 

7,292 

Aggregate  costs 

Total 

$5,322,632 

$2,583,281 

$2,739,350 

Outpatient 

2,260,564 

1,046,217 

1,214,346 

Inpatient 

3,062,068 

1,537,064 

1,525,004 

Costs  per  patient 

Total 

441 

541 

376 

Outpatient 

187 

219 

167 

Inpatient 

254 

322 

209 

NOTE:  All  costs  are  in  1999  dollars  that  were  estimated  by  adjusting  1998 
unit  cost  estimates  by  an  inflation  factor  of  1.4  percent. 


Variations  in  Costs  Across  Facilities 

Within  each  of  the  demonstration  and  control  groups,  we  found  sub¬ 
stantial  variation  among  individual  MTFs  in  baseline  levels  of  costs  of 
care  per  patient  and  changes  in  those  costs  between  study  years.  In 
Tables  5.4  and  5.5,  we  present  the  total  and  per-patient  costs  for  our 
sample — asthma  patients  enrolled  at  the  demonstration  or  control 
MTFs.  Estimated  costs  for  enrollees  at  the  demonstration  sites  are  in 
Table  5.4  and  those  for  enrollees  at  control  sites  are  in  Table  5.5. 

Total  costs  per  MTF  enrollee  varied  more  widely  across  the 
demonstration  sites  in  the  baseline  year  (from  $262  to  $437  per 
enrollee)  than  in  the  second  study  year  (from  $298  to  $353  per 
enrollee).  These  sites  also  varied  somewhat  in  the  share  of  total  costs 
per  enrollee  that  were  attributable  to  outpatient’s  costs. 


Table  5.4 

Estimated  Costs  of  Outpatient,  Inpatient,  and  Total  Services  for  Asthma  Patient  MTF  Enrollees  at  the 
Demonstration  Sites,  Study  Years  One  and  Two 


Outpatient  Services 

Inpatient  Services 

All  Services 

Number  of 
Patients 

Total  Cost 

Cost  per 
Patient 

Total  Cost 

Cost  per 
Patient 

Total  Cost 

Cost  per 
Patient 

Study  year  one 
Demo  1 

1,176 

233,286 

198 

280,824 

239 

514,110 

437 

Demo  2 

1,159 

231,745 

200 

243,457 

210 

475,202 

410 

Demo  3 

1,715 

375,397 

219 

213,692 

125 

589,089 

343 

Demo  4 

581 

89,100 

153 

63,236 

109 

152,336 

262 

Study  year  two 
Demo  1 

1,059 

150,837 

142 

222,817 

210 

373,654 

353 

Demo  2 

1,043 

171,563 

164 

184,938 

177 

356,501 

342 

Demo  3 

1,628 

401,751 

247 

195,171 

120 

596,922 

367 

Demo  4 

559 

94,727 

169 

71,991 

129 

166,718 

298 

NOTE:  All  costs  are  in  FY  1999  dollars  that  were  estimated  by  adjusting  1998  unit  cost  estimates  by  an  inflation  factor 


of  1 .4  percent. 
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Table  5.5 

Estimated  Costs  of  Outpatient,  Inpatient,  and  Total  Services  for  Asthma  Patient  MTF  Enrollees  at  the 
Control  Sites,  Study  Years  One  and  Two 


Outpatient  Services 

Inpatient  Services 

All  Services 

Number  of 
Patients 

Total  Cost 

Cost  per 
Patient 

Total  Cost 

Cost  per 
Patient 

Total  Cost 

Cost  per 
Patient 

Study  year  one 

Control  1 

1,635 

441,699 

270 

251,998 

154 

693,697 

424 

Control  2 

785 

128,353 

164 

92,536 

118 

220,889 

281 

Control  3 

736 

83,958 

114 

60,268 

82 

144,226 

196 

Control  4 

2,621 

322,024 

123 

336,867 

129 

658,891 

251 

Control  5 

1,747 

253,835 

145 

352,049 

202 

605,884 

347 

Control  6 

815 

114,277 

140 

144,283 

177 

258,560 

317 

Study  year  two 

Control  1 

1,280 

336,025 

263 

175,308 

137 

511,333 

399 

Control  2 

659 

94,973 

144 

84,945 

129 

179,918 

273 

Control  3 

595 

65,560 

110 

50,007 

84 

115,567 

194 

Control  4 

2,050 

240,707 

117 

393,547 

192 

634,254 

309 

Control  5 

1,558 

185,558 

119 

254,375 

163 

439,933 

282 

Control  6 

889 

140,570 

158 

58,621 

66 

199,191 

224 

NOTE:  All  costs  are  in  FY  1999  dollars  that  were  estimated  by  adjusting  1998  unit  cost  estimates  by  an  inflation  factor 
of  1 .4  percent. 
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The  individual  MTFs  in  the  control  group  also  varied  in  both 
the  levels  of  total  costs  of  care  per  enrollee  and  the  extent  to  which 
those  costs  changed  from  baseline  to  study  year  two  (Table  5.5). 
Costs  in  study  year  one  ranged  from  $196  to  $424  per  patient,  and 
those  for  study  year  two  ranged  from  $194  to  $399  per  patient.  The 
per-capita  costs  for  some  of  the  control  MTFs  declined  in  study  year 
two  while  those  for  other  control  MTFs  increased. 


Summary 

•  The  RAND  analysis  found  no  changes  in  the  demonstration 
MTFs’  performance  on  the  six  clinical  practice  indicators  we 
tracked  that  could  be  attributed  to  introduction  of  the  asthma 
practice  guideline  in  the  demonstration  sites. 

•  All  three  indicators  for  use  of  asthma  medications  declined  from 
the  first  to  second  study  year,  which  was  the  reverse  of  the 
hypothesized  direction  of  change. 

Service  use  indicators  did  not  show  change  that  could  be  attrib¬ 
uted  to  the  guideline.  Outpatient  visit  rates  for  the  demonstra¬ 
tion  MTFs  did  not  change  between  years  one  and  two,  although 
we  observed  seasonal  variations  for  all  three  study  groups.  Emer¬ 
gency  room  use  rates  declined  in  the  second  study  year  for  the 
target  demonstration  group,  but  this  effect  could  not  be  attrib¬ 
uted  to  use  of  the  guideline  because  rates  also  declined  for  the 
other  two  groups.  Hospitalization  rates  did  not  change  over  time 
for  the  target  demonstration  or  control  groups,  and  a  decline  in 
the  average  rate  observed  for  the  other  demonstration  group 
could  not  be  attributed  to  use  of  the  guideline  because  the  sites 
were  not  implementing  it  in  those  clinics. 

•  The  cost  analysis  indicated  that  a  substantial  share  of  MTF  costs 
for  asthma  patients  during  the  study  years  were  incurred  for 
patients  not  enrolled  at  the  MTFs,  especially  for  use  of  inpatient 
services. 

•  The  cost  analysis  also  found  a  decrease  between  the  two  study 
years  in  per-patient  costs  for  enrolled  patients  at  the  demonstra- 
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tion  MTFs.  Costs  also  decreased  for  the  control  MTFs  but  by  a 
smaller  percentage.  It  should  be  noted  that  based  on  our  analy¬ 
ses,  which  did  not  control  for  variation  of  other  variables  that 
might  impact  costs  of  care  (such  as  level  of  service  use,  etc),  it  is 
not  possible  to  conclude  that  the  observed  changes  costs  noted 
above  can  be  attributed  to  the  implementation  of  the  asthma 
guideline. 


CHAPTER  SIX 

Synthesis  of  Findings  from  the  Demonstration 


In  this  chapter,  we  synthesize  the  factors  influencing  the  successes  and 
limitations  of  the  asthma  guideline  demonstration.  We  begin  hy  dis¬ 
cussing  the  implications  of  our  findings  regarding  the  implementa¬ 
tion  process,  including  an  evaluation  of  how  well  the  demonstration 
performed  on  the  six  critical  success  factors  presented  in  Chapter  One 
of  this  report.  Next  we  discuss  the  implications  of  our  analysis  of  the 
effects  of  the  asthma  guideline  implementation  on  the  clinical  prac¬ 
tice  indicators.  We  will  also  discuss  the  implications  of  our  cost 
analysis.  Then  we  will  describe  data  issues  that  emerged  from  the 
demonstration  that  are  likely  to  affect  other  MTF  guideline  imple¬ 
mentation  and  monitoring  efforts.  Finally,  we  provide  our  recom¬ 
mendations. 


Findings  on  the  Implementation  Process 

Implementing  the  Guideline  Practices 

In  assessing  lessons  from  this  demonstration  regarding  the  progress  of 
MEDCOM  and  the  MTFs  in  practice  guideline  implementation,  it  is 
necessary  to  look  at  the  MEDCOM  and  MTE  activities  separately. 
Although  the  MEDCOM  staff  already  had  experience  working  with 
guideline  implementation  activities  as  the  asthma  demonstration 
started,  the  participating  MTE s  still  were  new  to  the  process  and  were 
learning  new  methods  during  the  demonstration. 
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As  MEDCOM  began  work  on  the  asthma  guideline  demonstra¬ 
tion,  the  MEDCOM  staff  already  had  strengthened  their  capabilities 
based  on  lessons  from  the  earlier  low  back  pain  guideline  demonstra¬ 
tion,  and  the  MEDCOM  organizational  infrastructure  had  been  fur¬ 
ther  developed  to  better  support  the  MTEs.  The  number  of 
MEDCOM  staff  had  been  expanded,  it  had  a  developed  Web  site, 
and  it  had  well-established  procedures  for  developing  guideline  tool¬ 
kits.  As  the  asthma  demonstration  started,  this  infrastructure  was  in 
full  operation  and  the  low  back  pain  guideline  was  being  imple¬ 
mented  across  the  AMEDD  system.  This  new  MEDCOM  capability 
yielded  more  effective  start-up  and  continuing  support  for  the  MTEs 
participating  in  the  asthma  demonstration,  which  was  readily  seen  by 
our  direct  observation  of  its  activities  as  well  as  feedback  from  the 
MTEs  during  the  site  visits. 

At  the  MTE  level,  the  process  evaluation  documented  that  par¬ 
ticipating  MTEs  made  observable  progress  in  introducing  changes  to 
achieve  evidence-based  clinical  practices  as  specified  in  the  asthma 
guideline.  By  the  end  of  the  ten-month  demonstration  period,  how¬ 
ever,  the  MTEs  still  were  working  on  getting  new  practices  into  place 
in  the  selected  clinics  they  chose  to  work  on  the  guideline,  and  none 
had  progressed  to  implementing  these  practices  across  all  relevant 
clinics.  Eack  of  experience  appeared  to  slow  their  initial  implementa¬ 
tion  pace,  but  observable  progress  in  their  activities  occurred  between 
the  first  and  second  site  visits.  The  MTEs  had  some  notable  success  in 
putting  into  place  new  capabilities  (case  management,  expanded 
patient  education,  list  of  asthma  patients,  monitoring  of  asthma  met¬ 
rics)  and  procedures  (formal  referrals  from  emergency  room  to  PCM, 
to  patient  education,  and  to  spirometry)  in  support  of  the  asthma 
guideline.  However,  they  were  not  equally  successful  in  gaining  the 
commitment  of  providers  to  use  these  new  or  expanded  resources, 
often  because  providers  thought  that  use  of  the  new  practices  would 
increase  their  workload.  In  addition,  none  of  the  sites  was  able  to 
address  the  telephone  triage  component  of  the  guideline. 
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Six  Critical  Success  Factors 

Research  on  practice  guideline  implementation  has  documented  that 
a  commitment  to  the  implementation  process,  including  use  of  mul¬ 
tiple  interventions,  is  required  to  achieve  desired  changes  to  clinical 
practices.  In  Chapter  One,  we  presented  six  critical  success  factors  for 
making  lasting  changes  in  MTFs’  clinical  and  administrative  pro¬ 
cesses.  We  discuss  here  the  extent  to  which  this  demonstration 
realized  these  success  factors  and  implications  for  progress  in  imple¬ 
menting  practice  improvements. 

•  Leadership  commitment  at  the  MTF,  regional,  and  com¬ 
mand  levels.  This  demonstration  had  somewhat  more  positive 
support  from  the  leadership  of  the  participating  MTFs  than  had 
been  provided  in  the  earlier  low  hack  pain  guideline  demonstra¬ 
tion,  hut  attitudes  hy  leadership  at  the  regional  and  system  levels 
tended  to  he  mixed.  For  the  MTF  that  already  had  been  work¬ 
ing  with  an  asthma  guideline  before  the  demonstration  started, 
the  MTF  leadership  was  committed  to  the  process,  and  its  main 
concern  was  how  to  build  on  their  earlier  work  most  effectively 
as  the  DoD/VA  guideline  went  into  the  field.  For  the  other 
three  MTFs,  the  command  teams  generally  supported  the 
implementation  teams  as  they  undertook  the  implementation 
process,  but  they  did  not  establish  this  work  as  a  key  MTF  pri¬ 
ority.  At  the  MEDCOM  level,  leaders  had  differing  views  about 
the  guideline,  and  negative  feedback  on  the  guideline’s  value  and 
effectiveness  was  expressed  throughout  the  study  period. 

•  Monitoring  of  progress.  The  monitoring  track  record  during 
the  asthma  guideline  demonstration  was  mixed.  The  MTF 
teams  focused  on  monitoring  the  extent  to  which  the  new  clini¬ 
cal  practices  they  had  introduced  were  being  used  (e.g.,  docu¬ 
mentation  of  asthma  severity  in  patients’  charts).  This  focus  is 
appropriate  when  first  beginning  to  work  with  new  practices 
because  it  allows  verification  that  these  practices  actually  are 
becoming  an  integral  part  of  clinic  processes.  The  MTFs  also 
performed  chart  reviews  to  collect  the  needed  data  to  assess 
status  on  DoD/VA  asthma  measures.  However,  no  monitoring 
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of  asthma  metrics  was  being  performed  by  MEDCOM,  with  the 
exception  of  the  analyses  performed  for  this  evaluation.  For  fully 
effective  performance  tracking,  monitoring  also  needs  to  be 
done  at  the  system  level. 

•  Guidance  and  support  to  the  MTFs  by  MEDCOM.  By  the 

time  the  asthma  guideline  demonstration  began,  MEDCOM 
had  expanded  its  staffing  and  other  resources,  and  we  observed 
MEDCOM  staff  providing  regular  policy  guidance  and  techni¬ 
cal  support  during  the  demonstration  to  help  the  MTFs  imple¬ 
ment  practice  improvements  for  asthma  care.  The  MTF  teams 
reported  that  MEDCOM ’s  committed  support  was  helpful  to 
them  and  they  were  pleased  to  have  it.  This  commitment  estab¬ 
lishes  a  foundation  on  which  to  build  as  future  practice  guide¬ 
lines  are  implemented  across  all  MTFs  in  the  Army  health 
system. 

•  Guideline  champions  who  are  opinion  leaders.  The  partici¬ 
pating  MTFs  identified  well-respected  physicians  to  serve  as 
guideline  champions  for  the  asthma  demonstration,  and  these 
physicians  showed  a  commitment  to  leading  the  implementa¬ 
tion  activities  for  their  facilities.  However,  the  champions  could 
only  make  a  time-limited  commitment  to  the  initiative,  and 
later  in  the  demonstration,  they  reported  they  were  tiring  of  the 
concentrated  effort  or  had  to  turn  their  attention  to  other  priori¬ 
ties.  This  finding  repeated  the  experience  of  the  champions  par¬ 
ticipating  in  the  low  back  pain  guideline  demonstration. 

•  Resource  support  for  champions.  The  MTF  commanders 
authorized  and  supported  the  champions  in  leading  the  imple¬ 
mentation  of  the  asthma  guideline.  However,  the  only  tangible 
resource  support  for  the  champions  was  for  their  attendance  at 
the  kickoff  conference.  The  champions  neither  received  dedi¬ 
cated  staff  time  for  their  implementation  activities  nor  were  they 
relieved  of  any  of  their  other  responsibilities.  As  a  result,  the 
champions  performed  the  implementation  work  in  addition  to 
their  regular  workload,  which  contributed  to  their  inability  and 
unwillingness  to  sustain  the  champion  role  on  a  regular  basis. 
The  facilitators  designated  by  the  MTF  commander  provided 


Synthesis  of  Findings  from  the  Demonstration  1 1 3 


some  staff  support  to  the  MTF  champions,  and  much  of  this 
support  role  was  part  of  the  facilitators’  regular  jobs  in  the  MTF 
quality  management  offices. 

Lack  of  resource  support  to  the  champions  appears  to  have  con¬ 
tributed  to  the  slow  start  experienced  by  many  MTFs  in  imple¬ 
menting  their  action  plans.  It  may  also  have  been  a  factor  in  the 
MTFs’  inability  to  achieve  the  extent  of  practice  improvements 
that  was  intended. 

•  Institutionalization  of  new  practices.  At  the  time  of  the  last 
process  evaluation  site  visit,  the  participating  MTFs  had  made 
progress  in  introducing  improved  asthma  management  practices 
in  some  of  their  primary  care  clinics,  but  they  had  not  yet 
achieved  sustainable  practices  in  those  clinics.  None  of  them  had 
yet  begun  to  extend  the  new  practices  into  other  clinics  to 
achieve  guideline  implementation  throughout  the  entire  facility. 
To  support  continued  efforts  to  achieve  sustainability,  the 
MTFs  were  developing  regular  education  sessions  for  providers, 
clinic  staff,  and  newcomers  to  the  MTF. 


Effects  of  Implementing  Practice  Improvements 

Outcome  Measures 

Our  analysis  of  the  effects  of  use  of  the  asthma  practice  guideline  on 
practices  found  no  changes  in  the  clinical  practice  indicators  that  we 
tracked.  All  three  indicators  for  use  of  asthma  medications  declined 
from  the  first  to  second  study  year,  which  was  the  reverse  of  the 
hypothesized  direction  of  change.  Outpatient  visit  rates  for  the  dem¬ 
onstration  MTFs  did  not  change  from  year  one  to  year  two.  How¬ 
ever,  we  observed  seasonal  variations  for  each  group  of  MTFs  in  the 
study — the  demonstration  sites,  the  targeted  clinics  within  the  dem¬ 
onstration  sites,  and  the  control  sites. 

We  found  no  change  in  emergency  room  visit  rates  and  hospi¬ 
talization  rates.  These  findings  suggest  that  either  MTFs  have  not  yet 
implemented  sufficient  practice  improvements  to  achieve  changes  in 
the  measures,  that  the  changes  have  not  had  sufficient  time  to  pro- 
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duce  the  expected  improvements,  or  that  the  measures  used  did  not 
adequately  capture  the  full  extent  of  practice  changes  that  have 
occurred. 

Potential  Contributing  Factors 

Several  factors  may  potentially  have  contributed  to  these  results. 

Insufficient  Authority  of  New  Program.  By  definition,  a  demon¬ 
stration  is  the  first  field  attempt  to  work  with  new  practices,  and 
therefore  it  lacks  the  full  authority  of  a  program  officially  imple¬ 
mented  across  an  entire  system. 

Length  of  Study  Period.  In  addition,  one  year  is  a  short  time  in 
which  to  expect  to  find  meaningful  changes  in  performance  on  many 
of  the  measures  of  interest  for  quality  improvement  programs.  For 
the  asthma  guideline,  we  would  first  expect  to  observe  changes  in  the 
processes  and  procedures  used  to  deliver  care  because  that  is  where 
the  MTFs  focused  their  implementation  strategies.  As  the  guideline  is 
institutionalized,  we  should  be  able  to  observe  changes  in  the  clinical 
practice  indicators. 

Interaction  of  Effects  in  Response  to  Practice  Changes.  The  lack 
of  observed  improvement  could  also  have  resulted  in  part  from  an 
interaction  of  opposing  effects  in  response  to  practice  changes.  For 
example,  better  classification  of  asthma  severity  could  result  in  more 
patients  being  reclassified  at  the  “mild  intermittent”  level,  and  the 
accompanying  reduction  of  medications  might  offset  the  effect  of 
reclassifications  to  higher  severity  levels. 

Insufficiency  of  the  Measures  Used.  Because  the  service-use 
measures  are  limited  to  counts  of  events,  they  cannot  detect  changes 
in  practice  that  occur  within  each  of  those  events  and  in  related  ser¬ 
vices.  Other  indicators  more  specific  to  the  practices  MTFs  did 
change  might  have  shown  improvements.  For  example,  data  provided 
by  the  MTFs  from  their  medical  chart  abstractions  suggested  that 
improvements  had  been  achieved  in  many  areas,  including  documen¬ 
tation  of  asthma  severity,  regular  use  of  spirometry,  and  development 
of  patient  action  plans.  MTFs  also  reported  improvements  in  pro¬ 
viding  treatment  appropriate  for  the  level  of  asthma  severity  recorded. 
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If  accurate,  these  data  would  represent  important  first  steps 
toward  widespread  application  of  more  active  care  management  for 
asthma  patients.  However,  as  noted  earlier  in  the  report,  we  did  not 
have  much  confidence  in  the  data  the  MTFs  developed  from  chart 
abstractions  because  of  small  samples,  missing  charts,  and  differences 
in  methods  across  MTFs,  all  of  which  can  introduce  bias  into  their 
results. 

Related  measurement  factors  might  also  explain  why  we  did  not 
see  the  expected  changes  in  the  use  of  asthma  medications.  Achieving 
a  change  in  use  of  these  medications  involves  three  steps,  at  any  of 
which  the  process  may  break  down.  First,  the  provider  must  prescribe 
the  medication  as  appropriate  for  the  asthma  severity.  Second,  the 
patient  must  fill  the  prescription,  which  will  be  more  likely  to  occur  if 
the  patient  is  well  educated  on  how  the  medication  is  part  of  the 
management  of  his  or  her  asthma,  how  the  medication  effects  a 
reduction  in  exacerbations,  and  how  to  manage  side  effects.  Finally, 
the  patient  must  actually  take  the  medication  as  instructed  by  the 
provider.  Failures  in  the  first  two  steps  will  be  captured  in  the  analysis 
of  pharmacy  data.  Failure  in  the  last  step  will  not  be  captured  but 
could  affect  rates  of  emergency  room  visits  and  hospitalizations 
because  the  medications  cannot  help  prevent  exacerbations  if  the 
patient  is  not  taking  them. 

Inadequate  Emphasis  on  the  Role  of  Self-Care  by  the  Patient. 

The  implementation  or  the  measures  used  to  evaluate  implementa¬ 
tion  effects  also  might  not  give  adequate  attention  to  the  role  of  the 
patient  in  his  or  her  asthma  care.  The  active  involvement  of  the 
patient  is  critical  for  successful  management  of  his  or  her  asthma. 
Patients  typically  do  not  perceive  asthma  to  be  life  threatening, 
although  it  certainly  has  an  effect  on  their  quality  of  life  and  can  put 
them  at  risk  of  increased  morbidity  or  mortality.  To  the  extent  that 
patients  do  not  comply  with  self-care  plans,  their  inaction  may  con¬ 
tribute  to  lack  of  observed  effects  of  the  guideline  on  such  indicators 
as  emergency  room  visits  or  hospitalizations. 

The  demonstration  sites  reported  little  or  no  progress  in 
increasing  attendance  at  patient  education  sessions,  which  were 
intended  to  help  patients  become  active  participants  in  their  care. 
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Information  obtained  during  the  process  evaluation  highlighted 
inadequacies  in  patient  education  activities,  including  problems  with 
the  design  of  the  education  programs  themselves,  limited  receptivity 
of  providers  to  referring  patients  to  education,  and  lack  of  increase  in 
attendance  in  the  programs.  In  addition,  none  of  the  measures  used 
by  either  the  demonstration  MTFs  or  the  RAND  analysis  directly 
addresses  the  actions  taken  by  the  patient  for  self-care  and  appropriate 
use  of  prescription  asthma  medications. 

Costs 

While  we  did  not  find  changes  in  any  of  the  indicators  tracked,  we 
did  find  evidence  of  lower  costs  for  those  MTFs  that  had  imple¬ 
mented  the  guideline  compared  to  those  who  did  not.  These  con¬ 
flicting  results  highlight  the  difficulty  of  isolating  the  many  factors 
that  affect  outcomes.  It  is  quite  possible  that  factors  other  than  the 
MTFs’  guideline  implementation  actions  contributed  to  the  observed 
cost  reductions.  For  example,  it  is  possible  that  changes  in  outpatient 
service  mix  or  in  the  intensity  of  care  during  hospitalizations  led  to 
lower  costs  for  care  of  enrollees  in  the  demonstration  MTFs. 

After  new  care  management  methods  are  in  place  for  a  while,  it 
will  be  important  to  track  inpatient  use  rates  and  costs  to  identify 
trends  and  longer-term  effects.  Once  several  years  of  cost  information 
are  available,  it  will  be  possible  to  discern  trends  related  to  practice 
changes  from  normal  year-to-year  fluctuations  in  health-care  needs. 

The  cost  analysis  also  found  that  a  substantial  share  of  MTF 
costs  for  asthma  patients  during  the  study  were  incurred  for  patients 
not  enrolled  at  the  MTFs,  especially  for  use  of  inpatient  services.  This 
finding  has  implications  for  how  best  to  serve  these  nonenrollees, 
considering  both  issues  of  care  management  for  episodic  users  of  the 
facilities  and  efficient  use  of  outpatient  and  inpatient  resources. 


Data  Issues 

Accurate  assessment  of  MTFs’  performance  in  implementing  treat¬ 
ment  guidelines  requires  the  capability  to  routinely  generate  accurate 
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and  reliable  data  on  the  indicators  monitored.  Pertinent  to  this  need, 
we  identified  three  critical  data  issues  that  need  to  be  addressed: 


•  Inconsistent  coding  of  diagnoses  and  procedures.  Effective 
monitoring  of  performance  in  treating  asthma  (or  some  other 
condition)  requires  consistent  coding  of  diagnoses  and  proce¬ 
dures  in  the  outpatient  encounter  records.  MEDCOM  has 
established  standard  codes  for  asthma,  but  at  the  time  of  the 
demonstration  these  codes  had  just  been  introduced  and  were 
not  used  consistently  by  the  demonstration  MTE s. 

•  Unavailable  data.  At  the  system  level,  data  needed  to  calculate 
many  indicators  (e.g.,  laboratory  or  radiology  data)  were  incom¬ 
plete,  were  obtained  from  separate  data  extraction  processes  of 
varying  quality,  or  were  not  currently  available. 

•  Absence  of  an  asthma  registry.  The  Army  health  system  lacks 
any  sort  of  centralized  registry  to  provide  complete  information 
on  all  asthma  patients  in  the  system  that  can  be  accessed  by 
MTEs  wherever  they  may  be.  In  the  absence  of  this  data 
resource,  asthma  patients  might  not  be  identified  or  information 
on  their  past  care  and  asthma  status  might  be  lost  as  personnel 
and  their  families  move  to  new  locations. 


Recommendations 

All  of  these  uncertainties  reinforce  both  the  difficulty  of  documenting 
effects  of  practice  improvements  on  asthma  care  outcomes  and  the 
need  to  continue  to  track  performance  longitudinally.  Through  the 
accumulation  of  data  over  time,  significant  trends  can  begin  to  be 
detected,  and  the  regular  feedback  of  data  to  the  MTEs  from  this 
process  will  help  them  to  determine  where  they  need  to  focus  future 
efforts. 

Ultimately,  a  practice  guideline  cannot  be  said  to  be  imple¬ 
mented  until  lasting  changes  in  practices  are  made.  Yet  all  of  the 
MTEs  participating  in  this  demonstration  had  difficulty  integrating 
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the  new  practices  into  normal,  ongoing  MTF  clinic  operations.  This 
finding  highlights  the  need  for  focused  attention  hy  the  leadership  of 
MEDCOM  and  the  MTFs  to  communicate  clearly  that  achieving 
best  practices  is  a  system  priority.  MEDCOM  also  needs  to  continue 
to  support  and  reinforce  the  MTFs’  efforts  hy  providing  technical 
support  and  establishing  an  effective  monitoring  system  to  provide 
feedback  to  the  MTFs  on  their  progress. 

We  summarize  here  our  recommendations  for  improving  the 
implementation  of  the  asthma  guideline. 

•  MEDCOM  should  establish  consistent  monitoring  standards 
for  performance  metrics.  Centralized  monitoring  standards 
will  be  important  to  ensure  that  consistent  data  are  being  col¬ 
lected  and  reported.  Only  with  this  consistency  can  MEDCOM 
and  the  MTFs  have  confidence  in  the  accuracy  of  observed  dif¬ 
ferences  across  MTFs  and  changes  over  time  in  the  indicators 
being  tracked.  To  achieve  this  consistency,  standardized  coding 
for  patient  status  or  procedures  will  need  to  be  implemented 
effectively  across  the  Army  MTFs.  Some  progress  has  been  made 
in  the  demonstrations  in  defining  standardized  codes  to  identify 
patients  (e.g.,  low  back  pain  patient  codes),  status  of  condition 
(e.g.,  asthma  severity),  and  specific  procedures  (e.g.,  foot  exams 
for  diabetes).  However,  these  codes  have  not  been  used  consis¬ 
tently  by  all  MTFs,  with  the  result  that  the  data  aggregated  at 
the  system  level  for  these  variables  cannot  be  trusted. 

MEDCOM  will  need  to  consider  data  and  technology  issues 
when  it  makes  decisions  regarding  how  to  establish  an  effective 
system-level  information  base  with  complete  and  consistently 
defined  data  on  the  performance  indicators  being  monitored. 
One  option  would  be  to  establish  a  centralized  system  to  collect 
the  data  directly  from  automated  data  systems,  perform  analyses 
in  the  central  office,  and  generate  trend  reports  to  the  MTFs. 
Another  option  would  be  for  MTFs  to  collect  and  analyze  data 
locally  and  then  report  to  MEDCOM,  which  would  then  aggre¬ 
gate  the  individual  MTF  results  into  trend  reports.  To  support 
this  approach,  MEDCOM  would  need  to  define  consistent 


Synthesis  of  Findings  from  the  Demonstration  1 1 9 


measurement  methods  and  standards  for  the  MTFs  to  use,  and 
it  would  have  to  perform  regular  audits  for  measurement  consis¬ 
tency  to  ensure  the  integrity  of  the  data  for  effective  perfor¬ 
mance  monitoring. 

MEDCOM  should  work  with  the  MTFs  to  establish  per¬ 
formance  objectives  on  the  asthma  metrics.  Three  of  the  four 
sites  monitored  the  asthma  metrics  during  the  demonstration. 
The  implementation  teams  did  not  feed  this  performance 
information  hack  to  the  providers  and  other  clinic  staff, 
however,  so  they  would  have  empirical  knowledge  of  their  per¬ 
formance  on  key  aspects  of  care.  Neither  did  they  use  this  infor¬ 
mation  to  hold  clinics  or  providers  accountable  for  their 
performance.  Both  of  these  steps  are  essential  to  effective  use  of 
monitoring  for  improving  clinical  care  performance.  To  ensure 
that  performance  information  improves  clinical  practices,  moni¬ 
toring  of  the  asthma  metrics  should  he  integrated  into  the 
MTFs’  quality  management  or  peer  review  programs,  and  the 
MTF  commanders  should  review  processes  and  results  regularly. 
MEDCOM  should  develop  software  programs  necessary  to 
allow  the  MTFs  to  retrieve  CHCS  and  ADS  data.  MTFs  cur¬ 
rently  have  difficulty  retrieving  ADS  and  CHCS  data  for  use  in 
the  monitoring  process.  To  address  this  difficulty,  the  MTFs 
requested  that  MEDCOM  provide  them  with  the  “ad  hoc” 
software  programs  needed  to  extract  the  data.  Accurate  coding 
of  asthma  diagnosis  and  severity  in  the  ADS  data  is  also  required 
to  identify  these  patients  for  analysis.  MEDCOM  established  a 
standard  code  for  asthma  but  needs  to  provide  specific  instruc¬ 
tions  on  coding  responsibilities  and  timeliness,  as  well  as  effec¬ 
tive  techniques  for  sampling  and  retrieving  information  from 
patient  charts  for  auditing  purposes. 

As  MEDCOM  monitors  the  asthma  metrics  across  MTFs,  it 
needs  to  identify  where  improvements  in  quality  and  consis¬ 
tency  of  care  are  needed.  The  MTFs  were  given  considerable 
flexibility  to  develop  implementation  strategies  that  reflected 
each  MTF’s  unique  capabilities  and  prior  experiences  with 
asthma  treatment  improvement  projects  and  guidelines.  Each 
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MTF  chose  to  emphasize  different  components  of  the  guideline 
and  differed  in  how  widely  they  implemented  the  guideline  in 
their  TMCs  and  various  other  clinics.  We  believe  this  flexibility 
helps  to  ensure  that  each  team  can  address  the  clinical  practices 
most  in  need  of  improvement  at  its  own  MTF.  On  the  other 
hand,  such  flexibility  involves  some  risk  that,  when  a  team  meets 
staff  resistance,  it  might  pursue  only  expedient  actions,  which 
would  slow  progress  toward  the  AMEDD  goal  of  achieving  con¬ 
sistent  practices  across  as  well  as  within  facilities.  By  continuing 
to  monitor  the  metrics  closely,  MEDCOM  can  determine 
whether  to  give  greater  direction  to  MTEs  with  regard  to  which 
aspects  of  the  guideline  are  to  be  emphasized  and  implemented 
uniformly. 

•  MEDCOM  needs  to  establish  clear  procedures  and  expecta¬ 
tions  for  the  use  of  forms.  Although  sites  were  told  that  the  use 
of  the  forms  provided  by  MEDCOM  was  voluntary,  partici¬ 
pants  at  some  of  the  MTEs  still  thought  that  use  of  the  forms 
was  mandatory.  Other  sites  chose  not  to  use  the  forms,  but  they 
did  not  provide  clear  alternative  methods  to  ensure  that  asthma 
diagnosis  and  treatment  were  being  documented  appropriately. 
As  a  result,  standardization  of  information  was  lacking  and 
uncertainty  arose  in  the  MTEs  about  the  procedural  steps  they 
should  have  been  taking.  MEDCOM  needs  to  clarify  its  expec¬ 
tations  concerning  the  use  of  forms,  particularly  for  patients 
with  multiple  conditions  for  which  two  or  more  practice  guide¬ 
lines  apply. 

•  MEDCOM  needs  to  further  define  the  role  of  patient  educa¬ 
tion  in  treatment  processes  of  chronic  conditions,  while 
MTEs  need  to  ensure  that  they  are  using  the  most  effective 
patient  education  techniques.  The  issue  of  patient  education 
has  increased  in  salience  for  AMEDD  because  many  of  the 
guidelines  it  has  implemented  are  for  chronic  conditions  that 
require  self-care  management  by  patients  for  effective  overall 
management  of  the  condition.  All  MTE  teams  participating  in 
the  demonstration  said  they  considered  patient  education  to  be 
an  important  component  in  improving  self-care  management  of 
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asthma,  and  they  sought  to  strengthen  both  the  content  of  their 
education  programs  and  procedures  for  referrals  to  those  pro¬ 
grams.  All  the  teams  reported  low  incidence  of  provider  referrals, 
however,  and  they  reported  that  providers  tended  to  he  skeptical 
about  the  effectiveness  of  education  provided  by  the  MTFs. 
They  also  found  it  difficult  to  motivate  patients  to  attend  classes 
and  take  responsibility  for  their  care.  MEDCOM  needs  to 
establish  clear  standards  for  patient  education  and  ensure  that 
MTFs  have  adequate  resources  and  tested  educational  methods. 
Part  of  the  problem  with  the  programs’  effectiveness  may  have 
stemmed  from  the  didactic  lecturing  approach  used.  MTFs 
should  adopt  a  collaborative,  problem-solving  approach,  which 
has  been  shown  more  effective  for  such  courses. 

MTFs  need  to  integrate  training  on  clinical  guidelines  into 
their  ongoing  education  for  current  personnel  and  into  the 
orientation  for  incoming  primary  care  providers  and  ancil¬ 
lary  staff.  At  the  start  of  the  demonstration,  the  participating 
MTFs  sought  to  introduce  and  train  primary  care  providers  on 
the  asthma  guideline  in  one  CME  session.  In  addition,  some  of 
the  sites  did  not  initially  train  the  ancillary  staff  on  the  new 
guideline,  expecting  that  it  would  be  sufficient  to  show  them  the 
encounter  form  and  tell  them  to  use  it.  However,  this  minimal¬ 
ist  approach  to  training  was  inadequate,  and  implementation 
teams  often  found  that  the  training  session  tended  to  turn  into  a 
discussion  of  whether  to  implement  the  guideline  rather  than 
how  to  implement  it.  To  train  all  primary  care  providers  to 
desired  levels  of  knowledge,  it  became  clear  that  multiple  and 
ongoing  training  sessions  would  be  required,  as  providers 
deployed  or  rotated  in  and  out  of  the  MTFs. 

MTFs  need  to  integrate  new  practices  into  normal  clinic 
operation — i.e.,  the  way  they  “do  business”  for  patient  care. 
A  practice  guideline  cannot  be  said  to  be  implemented  until 
such  lasting  changes  in  practices  are  made.  Reasons  for  this 
inability  to  “institutionalize”  the  new  practices  include  omission 
of  needed  actions  from  the  action  plans,  passive  leadership  sup¬ 
port,  resistance  to  change  by  providers  and  clinic  staff,  lack  of 
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clarity  on  staff  responsibilities  and  authority  to  make  changes, 
and  lack  of  accountability  for  achieving  desired  results.  T o  help 
MTFs  make  lasting  practice  improvements,  MEDCOM  needs 
to  communicate  clearly  that  achieving  best  practices  is  a  system 
priority,  and  it  should  continue  to  support  and  reinforce  the 
MTFs’  efforts  by  providing  technical  support  and  establishing 
an  effective  monitoring  system  to  track  and  provide  feedback  to 
the  MTFs  on  their  progress. 


APPENDIX  A 

Hypotheses  for  Effects  of  Improved  Asthma  Care 
Practices 


Initial  Asthma  Diagnosis 

The  first  set  of  measures  addresses  the  use  of  spirometry  to  establish 
(or  to  rule  out)  the  diagnosis  of  asthma  in  patients  ages  six  or  older. 
The  guideline  recommends  use  of  spirometry  during  an  initial  asthma 
work-up  and  annual  spirometry  for  those  with  confirmed  asthma.  We 
hypothesize  that  asthma  guideline  implementation  will 

•  increase  use  of  spirometry  among  “prohahle  adult  asthma 
patients”  and 

•  increase  use  of  spirometry  among  “possible  undiagnosed  adult 
asthma  patients.” 


Asthma  Follow-Up  Management,  Including  Medication 
Management 

These  measures  apply  to  probable  asthma  patients  and  their  encoun¬ 
ters.  We  hypothesize  that  implementation  of  the  guideline  will 

•  increase  the  percentage  of  patients  for  whom  asthma  severity 
level  is  assessed  during  asthma  visits, 

•  increase  use  of  annual  spirometry  for  asthma  patients,  and 

•  decrease  number  of  repeat  emergency  room  visits  and  hospital 
readmissions  for  asthma. 
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The  guideline  emphasizes  appropriate  use  of  medication  as  part 
of  follow-up  management.  We  hypothesize  that  implementation  of 
the  asthma  guideline  will  he  associated  with 

•  increased  prescribing  of  long-term  controller  medications  for 
persistent  asthma, 

•  increased  prescribing  of  quick-relief  medications  for  all  asth¬ 
matics, 

•  shift  in  medication  utilization  from  quick-relief  medication 
toward  long-term  controllers, 

•  decreased  percentage  of  asthma  hospitalizations  or  emergency 
room  visits  for  which  the  patient  has  no  previous  prescription 
for  a  long  term  controller, 

•  decreased  percentage  of  hospitalizations  or  emergency  room  vis¬ 
its  for  which  patients  have  no  prescription  for  long-term  con¬ 
trollers  immediately  after  discharge,  and 

•  increased  percentage  of  patients  referred  for  asthma  education. 


Exacerbation  Management 

Implementation  of  the  asthma  guideline  should  result  in 

•  increased  use  of  spirometry  associated  with  an  asthma  emergency 
room  visit, 

•  increased  use  of  oral  corticosteroids  associated  with  an  asthma 
emergency  room  visit, 

•  increased  number  of  asthma  cases  followed-up  after  a  hospital 
discharge,  and 

•  increased  use  of  pulse  oximetry  at  ER  visits. 


APPENDIX  B 


Evaluation  Methodology 


Process  Evaluation 

To  capture  the  full  dynamics  of  a  process  as  complex  as  practice 
guideline  implementation,  it  is  important  to  take  into  account  the 
roles  and  interactions  of  the  many  aspects  of  the  system  in  which  the 
guidelines  are  being  implemented.  Figure  B.l  is  a  diagram  of  relation¬ 
ships  among  the  different  levels  of  a  health-care  organization  during 
guideline  implementation,  the  stakeholders  involved,  and  the 
dynamics  of  the  implementation  process. 

Figure  B.1 

A  System  View  of  Guideline  Implementation 


EXTERNAL  ENVIRONMENT 


RAND  MG319-B.1 
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A  variety  of  stakeholders  need  to  be  considered  to  ensure  that 
individuals  involved  in  implementing  new  practices  anticipate  possi¬ 
ble  impacts  on  the  stakeholders  and  responses  that  might  be  expected 
from  them.  These  groups  include  treatment  program  leadership, 
middle  management,  the  clinical  and  administrative  staff  working 
with  program  residents,  and  the  clients  themselves.  The  implementa¬ 
tion  team  itself  consists  of  important  stakeholders  who  not  only  serve 
as  team  members  but  also  have  other  job  responsibilities. 

Information  was  collected  about  the  actions  involved  in  practice 
guideline  implementation  for  participating  MTFs,  the  dynamics  of 
the  change  process,  and  responses  of  participants  to  their  experiences 
with  the  process.  Similarities  and  differences  in  the  attitudes,  motiva¬ 
tions,  and  preferences  of  the  stakeholders  were  considered  as  the 
process  evaluation  information  was  collected  and  results  were  synthe¬ 
sized.  To  capture  changes  in  structures,  processes,  and  issues  as 
guideline  implementation  moved  forward,  site  visits  were  conducted 
to  collect  information  at  the  baseline  and  at  two  follow-up  times,  as 
shown  in  Table  B.  1 . 

A  formative  evaluation  approach  was  used  throughout  the 
implementation  process  and  evaluation.  In  addition  to  the  site  visits, 
we  used  routine  progress  reports  and  maintained  an  ongoing  com¬ 
munication  process  to  provide  a  structure  through  which  imple- 


Table  B.1 

Dimensions  Addressed  by  the  Process  Evaluation 


Baseline 

Month 

Three 

Month 

Nine 

Structure  and  organization 

X 

X 

X 

Culture  and  climate 

X 

X 

Current  practices 

X 

X 

X 

Environmental  context 

X 

X 

X 

Stakeholders'  attitudes 

X 

X 

X 

Implementation  plan 

X 

X 

Changes  in  clinic  processes 

X 

X 

AMEDD  support  systems 

X 

X 

Staff  involvement 

X 

X 

Patient  roles  and  reactions 

X 

X 

Monitoring  progress 

X 

X 

Effects  on  stakeholders 

X 

X 
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Table  B.2 

Dimensions  Addressed  by  the  Process  Evaluation  and  Data  Collection 
Methods 


Document 

Monitor 

Individual 

Focus 

Culture 

Materials 

Reports 

Interviews® 

Groups 

Survey 

Environmental  Context 

How  supportive  were  the  cul¬ 
ture  and  climate? 

How  did  the  culture  and  cli¬ 

X 

mate  change? 

What  were  the  other  factors 

X 

affecting  implementation? 

X 

X 

The  Implementation  Plan 

What  key  guideline  elements 
are  priorities? 

What  information  is  needed 

X 

X 

X 

X 

to  identify  priorities? 

How  is  guideline  team  orga¬ 

X 

X 

nized? 

X 

X 

X 

How  does  guideline  team 
operate? 

How  was  guideline  intro¬ 

X 

X 

duced  to  staff? 

X 

X 

X 

X 

Planned  Changes  to  Processes 

What  process  changes  did 

MTFs  identify? 

Which  changes  did  MTFs 

X 

X 

X 

X 

implement? 

X 

X 

X 

What  factors  supported  or 
slowed  changes? 

How  were  implementation 

X 

X 

X 

plans  changed? 

X 

X 

X 

X 

AMEDD  Systems  for  Implemen¬ 

tation 

What  help  was  received  from 

MEDCOM  on  implementa¬ 
tion? 

X 

X 

X 

How  useful  was  implementa¬ 
tion  toolkit? 

How  useful  were  KMN,  com¬ 

X 

X 

X 

munications? 

Did  MEDCOM  help  in  the 

X 

X 

X 

monitoring  role? 

X 

X 

X 

Clinical,  Administrative  Staff 
Effects 

What  were  the  attitudes  of 
MDs,  other  staff,  at  the 

start?  XXX 
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Table  B.2 — continued 


Document 

Monitor 

Individual 

Focus 

Culture 

Materials 

Reports 

Interviews® 

Groups 

Survey 

What  were  the  MD  and  other 

staff  roles  in  implementa¬ 
tion? 

X 

X 

Were  the  MDs  motivated  to 

adopt  new  practices? 

What  were  the  effects  of 

X 

X 

changes  on  MDs  and 
responses? 

What  were  the  effects  on 

other  staff  workload, 
demands? 

Roles  and  Reactions  of  Patients 

What  were  patients  responses 
to  changes  in  care? 

How  did  the  team  manage 

X 

X 

patient  reactions? 

How  helpful  were  patient 

X 

X 

education  materials? 

What  were  the  effects  on 

X 

X 

physician-patient  relation¬ 
ships? 

X 

X 

Measuring  Implementation  Progress 

What  were  the  indicators  MTF 

selected  for  monitoring?  X 

X 

What  was  the  MTF  data  sys¬ 
tem  for  monitoring?  X 

What  were  the  monitoring 

X 

X 

X 

lessons  and  actions  taken 

How  useful  was  monitoring 

X 

X 

X 

to  staff? 

X 

X 

^Individual  interviews  included  one-on-one  interviews  and  written  questionnaires 
completed  by  key  participants. 


menting  MTFs  could  get  assistance  from  each  other,  MEDCOM,  or 
RAND. 

Both  qualitative  and  quantitative  data  collection  methods  were 
used  in  the  process  evaluation  to  collect  information  on  a  set  of  ques¬ 
tions  that  cover  the  dimensions  shown  in  Table  B.l.  Shown  in  Table 
B.2  are  the  specific  topic  areas  covered  and  relevant  data  collection 
methods.  Interviews  and  focus  groups  with  the  implementation  team, 
providers  and  clinic  staff,  quality  management  staff,  and  other  par- 
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ticipants  yielded  information  on  the  dynamics  of  the  implementation 
process.  Focus  groups  were  conducted  with  three  types  of  partici¬ 
pants:  the  implementation  team,  providers,  and  other  clinic  staff.  Par¬ 
ticipants  in  each  stakeholder  group  were  asked  questions  regarding 
their  attitudes  toward  guideline  implementation,  how  they  worked 
with  the  practice  guideline,  how  they  were  affected  hy  the  implemen¬ 
tation  process,  and  issues  or  concerns  they  identified.  Semistructured 
interview  methods  were  used  for  all  interviews,  group  discussions, 
and  focus  groups,  working  from  lists  of  questions  to  cover  during 
each  session. 

A  brief  survey  regarding  stakeholders’  attitudes  toward  practice 
guidelines  and  quality  improvement  processes  was  administered  at 
baseline  and  the  final  site  visit.  The  survey  at  the  final  site  visit  also 
included  questions  about  education  received  on  the  guideline,  actions 
taken  to  implement  the  new  practices,  and  how  those  actions  affected 
providers  and  clinic  staff. 

Documents  and  materials  also  were  important  sources  of  infor¬ 
mation  for  the  process  evaluation.  These  included  written  infor¬ 
mation  about  the  MTF  structure  and  management,  policies  and 
procedures,  data  collection  and  monitoring,  and  materials  developed 
by  the  MTF  implementation  teams  as  they  prepared  and  carried  out 
their  action  plans  to  change  practices.  The  materials  provided  the 
primary  documentation  on  the  actions  planned  by  the  team,  changes 
made  to  clinic  processes,  resulting  events,  and  actions  taken  to  moni¬ 
tor  their  progress. 


Evaluation  of  Effects  (Outcomes) 

The  evaluation  of  the  effects  of  the  asthma  practice  guideline  demon¬ 
stration  was  designed  to  work  entirely  with  administrative  data.  Ide¬ 
ally,  these  data  would  have  included  a  master  enrollment  file  for 
beneficiaries  using  the  Army  MTFs  along  with  files  containing  data 
on  health  service  encounters.  Unfortunately,  although  a  master 
TRICARE  enrollment  file  contains  centralized  data  on  all  beneficiar¬ 
ies,  these  data  were  not  available  for  use  by  AMEDD.  Therefore,  we 
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had  to  work  entirely  with  data  from  the  health  service  encounters, 
including  SIDR,  SADR,  and  MTF  pharmacy  data  for  MTF  services 
and  HCSR  and  NMOP  data  for  network  provider  services.  An  exten¬ 
sive  process  of  data  extraction,  variable  derivation,  and  diagnostic 
analyses  was  carried  out  to 

1.  identify  correcdy  the  asthma  population  served  hy  the  Army 
MTFs  during  the  two-year  study  period, 

2.  select  the  study  sample  for  the  analysis  of  guideline  effects  on 
service  delivery,  consisting  of  the  subset  of  the  asthma  popula¬ 
tion  enrolled  at  any  demonstration  or  control  MTF,  and 

3.  establish  a  database  of  all  health-care  encounters  for  the  study 
sample. 

We  first  document  here  the  specific  steps  involved  in  the  data 
extraction  process  and  variable  specification  to  achieve  these  three 
work  products.  Then  we  summarize  the  codes  used  to  define  the  vari¬ 
ables  used  for  the  analysis  of  demonstration  effects. 


Overview  of  the  Data  Extraction  Process  for  the  Asthma 
Study 

Two  rounds  of  data  extraction  and  file  construction  were  performed 
in  collaboration  with  PASBA  to  establish  the  data  required  for  our 
analyses.  In  round  one,  we  extracted  data  necessary  to  identify  all  the 
asthma  patients  served  by  any  Army  MTFs  or  health  centers  during 
the  two  study  years,  and  we  established  a  data  file  containing  a  record 
for  each  patient  along  with  descriptive  data  on  them  that  could  be 
obtained  from  the  administrative  data.  In  round  two,  we  extracted 
data  on  all  encounters  for  the  subset  of  patients  enrolled  at  one  of  the 
MTFs  in  the  demonstration  or  control  groups,  which  included  their 
use  of  any  military  or  network  provider  services  for  any  reason. 
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Round-One  Data  Extraction  Specifications 

SADR  Data  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  SADR 
(MTF  outpatient)  files  that  meet  the  following  criteria: 

•  Encounter  date  between  October  1,  1998,  and  December  31, 

2000. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  of  493.xx  for  asthma  in  any  diagnostic  field  (we 
also  obtained  records  for  patients  with  bronchitis  diagnosis  but 
decided  to  exclude  these  patients  from  the  sample). 

•  Treated  at  any  Army  MTF  or  clinic  (all  Army  DMIS  IDs, 
including  clinics  and  TMCs). 

The  following  variables  were  extracted  from  the  SADR  file  data: 


Alternate  care  value 
Appointment  status  type 
Beneficiary  category 
Calendar  year  (created  by 
PASBA) 

ID  for  MTF  treating  the  patient 
Disposition  code 
Diagnosis  codes  I  through  4 
Encounter  date 


ID  for  MTF  where  enrolled  in 
Prime 

Patient  Zip  code  of  residence 
MEPRS  code  for  clinic  of  ser¬ 
vice 

MTF  location 
Patient  date  of  birth 
Patient  gender 

Sponsor  Social  Security  number 
(SSN) 

Family  member  prefix 


SIDR  Data  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  SIDR 
(MTF  inpatient)  files  that  meet  the  following  criteria: 


Admission  date  between  October  1,  1998,  and  December  31, 

2000. 
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•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  of  493.xx  for  asthma  in  any  diagnostic  field  (we 
also  obtained  records  for  patients  with  bronchitis  diagnosis  but 
decided  to  exclude  these  patients  from  the  sample). 

•  Treated  at  any  Army  MTF  (all  Army  DMIS  IDs,  including 
clinics  and  TMCs). 


The  following  variables  were  extracted  from  the  SIDR  file  data: 


Alternate  care  value 
Date  of  admission 
Admission  source 
Beneficiary  category 
Calendar  year  (created  by 
PASBA) 

Date  of  disposition  (discharge) 
Type  of  disposition 
ID  of  MTF  of  service 
MTF  name 

MEPRS  code  for  inpatient  unit 


Diagnosis  codes  1  through  8 
Date  of  disposition  (discharge) 
Patient  Zip  code  of  residence 
MTF  location 
Number  of  diagnoses 
Patient  date  of  birth 
Patient  gender 
Sponsor  SSN 
Family  member  prefix 


MTF  Pharmacy  Files 

Record  extraction  rules:  Keep  all  records  from  the  MTF  pharmacy  files 
that  meet  the  following  criteria: 


•  Beginning  date  of  service  between  October  1,  1998,  and 
December  31,  2000. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  At  least  two  prescriptions  from  the  list  of  asthma  medications 
(listed  below)  using  Therapeutic  Class  Code  and  National  Drug 
Code  provided  by  the  PEC. 

•  Eilled  a  prescription  at  any  Army  MTE  pharmacy  (all  Army 
DMIS  IDs,  including  clinics  and  TMCs). 
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The  following  variables  were  extracted  from  the  MTF  pharmacy 
file  data: 


Sponsor  SSN 

DEERS  dependent  suffix  (DDS) 
(family  member  code) 

Patient  date  of  birth 
Patient  age 
Gender 

Patient  category 
Enrollment  status 
Enrollment  DMIS  ID  code 
Eiscal  year 


Prescription  fill  date 
Generic  name 

Product  description  (name  and 
dosage) 

National  Drug  Gode 

Days  supply 

Quantity 

Dose  form 

Strength 

Refill  code 


Outpatient  Network  Provider  (HCSR)  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  HGSR 
Outpatient  files  that  meet  the  following  criteria: 


•  Beginning  date  of  service  between  October  1,  1998,  and 
December  31,  2000. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  of  493.xx  for  asthma  in  any  diagnostic  field  (we 
also  obtained  records  for  patients  with  bronchitis  diagnosis  but 
decided  to  exclude  these  patients  from  the  sample). 

•  Reside  in  the  catchment  area  of  an  Army  MTE  or  clinic.  Keep 
all  records  coded  equal  to  any  of  the  Army  facility  parent  DMIS 
IDs,  including  health  centers  and  MTEs.  This  will  pick  up  peo¬ 
ple  residing  in  the  parent  catchment  area  even  if  they  used  a 
freestanding  clinic  with  a  separate  DMIS  ID. 

The  following  variables  were  extracted  from  the  HGSR  outpa¬ 
tient  file  data: 
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Beginning  date  of  service 
Beneficiary  category 
Patient  catchment  area 
Calendar  year 

Defense  Enrollment  Eligibility 
Reporting  System  (DEERS) 
dependent  suffix 
Diagnoses  1  through  4 
Date  of  birth 
End  date  of  service 
Enrollment  DMIS  ID 


Enrollment  DMIS  parent 
region 

Enrollment  status 
Health  service  region 
Patient  age 
Patient  gender 

Patient  Zip  code  of  residence 
Primary  diagnosis 
Sponsor  SSN 
Sponsor  service  branch 


An  additional  variable  that  we  did  not  request  was  the  “type  of 
service”  variable,  which  identifies  the  specific  type  of  outpatient  visit 
for  which  the  HCSR  claim  was  submitted.  This  variable  would  have 
been  used  to  identify  visits  to  network  provider  emergency  rooms, 
which  should  have  been  included  in  the  indicator  for  emergency 
room  use  rates.  This  variable  should  be  included  in  any  future 
database  used  for  monitoring  this  indicator. 

Inpatient  Network  Provider  (HCSR)  Files 

Record  extraction  rules:  Keep  all  encounter  records  from  the  HCSR 
Inpatient  files  that  meet  the  following  criteria: 

•  Beginning  date  of  service  between  October  1,  1998,  and 
December  31,  2000. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  Have  a  code  of  493. xx  for  asthma  in  any  diagnostic  field  (we 
also  obtained  records  for  patients  with  bronchitis  diagnosis  but 
decided  to  exclude  these  patients  from  the  sample). 

•  Reside  in  the  catchment  area  of  an  Army  MTE  or  health  center. 
Keep  all  records  coded  equal  to  any  of  the  Army  facility  parent 
DMIS  IDs,  including  health  centers  and  MTEs.  This  will  pick 
up  people  residing  in  the  parent  catchment  area  even  if  they 
used  a  freestanding  clinic  with  a  separate  DMIS  ID. 
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The  following  variables  were  extracted  from  the  HCSR  inpa¬ 
tient  file  data: 


Admission  date 
Beginning  date  of  service 
Beneficiary  category 
Patient  catchment  area 
Calendar  year 
DEERS  dependent  suffix 
Discharge  status  code 
Primary  diagnosis 
Diagnoses  1  through  8 
Enrollment  DMIS  parent  region 


Date  of  birth 
End  date  of  service 
Enrollment  DMIS  ID 
Enrollment  status 
Patient  age 
Patient  gender 

Patient  Zip  code  of  residence 
Sponsor  SSN 
Health  service  region 
Sponsor  service  branch 


National  Mail  Order  Pharmacy  (NMOP)  Files 

Record  extraction  rules:  Keep  all  records  from  the  NMOP  files  that 
meet  the  following  criteria: 

•  Beginning  date  of  service  between  October  1,  1998,  and 
December  31,  2000. 

•  All  individuals  eligible  for  DoD  health-care  benefits,  including 
active-duty  personnel,  dependents,  or  retirees  in  any  service  as 
well  as  all  other  eligible  groups. 

•  At  least  two  prescriptions  from  the  list  of  asthma  medications 
(listed  below)  using  Therapeutic  Class  Code  and  National  Drug 
Code  provided  by  the  PEC.  Eirst  pull  records  using  Therapeutic 
Class  Code  and  then  pull  by  National  Drug  Code  from  that 
subset  of  records. 


were  extracted  from  the  NMOP  file 


The  following  variables 

data: 

Alternate  care  value 
Clinic  Zip  code  +  4 
Calendar  year 
DEERS  dependent  suffix 
(DDS) 


Patient  date  of  birth 
Patient  Zip  code  of  residence 
Prescription  transaction  date 
(date  filled,  not  date  posted) 
TRICARE  region  code 
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Enrollment  DMIS  ID 
Enrollment  parent  DMIS  ID 
Gender 

National  drug  code 

List  of  Asthma  Medications 

Clenbuterol  Hydrochloride 
Albuterol 
Albuterol  Sulfate 
Albuterol  Sulfate/Ipratropium 
Aminophylline 
Aminophylline-Ephedrine- 
amobarbital 

Beclomethasone  Dipropionate 
Bitolterol  Mesylate 
Budesonide 
Cromolyn  Sodium 
Dexamethasone  Sodium  Phos¬ 
phate 

Dyphylline 

Ephedrine  Hydrochloride 
Ephedrine  Sulfate 
Ephedrine-Potassium  Iodide 
Epinephrine 
Epinephrine  Bitartrate 
Elunisolide 
Elunisolide/Menthol 
Eluticasone  Propionate 
Guaifen/Dyphylline/Ephedrine/ 
PB 

Guaifen/Dyphylline/P- 

Ephedrine 

Guaifen/Theophylline  Anhyd/P- 
Ephedrine 

Guaifen/Theophylline/ 
Ephedrine/PB 
Guaifenesin/Dyphylline 
Guaifenesin/ Oxtriphylline 


Sponsor  service  branch 
Sponsor  SSN 
Therapeutic  class  code 


Guaifenesin/Theophylline 
Sodium  Glycinate 
Guaifenesin/Theophylline 
Guaifenesin/Theophylline/ 
Ephedrine 

Ipratropium  Bromide 
Isoetharine  Hydrochloride 
Isoproterenol  Hydrochloride 
Isoproterenol  Sulfate 
Eevalbuterol  Hydrochloride 
Metaproterenol  Sulfate 
Montelukast  Sodium 
Nedocromil  Sodium 
Oxtriphylline 
Pirbuterol  Acetate 
Salmeterol  Xinafoate 
Terbutaline  Sulfate 
Theophylline/Ephedrine/ 
Potassium  lodide/PB 
Theophylline/Isoproterenol/ 
EPD/Potassium  Iodide 
Theophylline  Anhydrous 
Theophylline/Dextrose  5%- 
water 

Theophylline/Ephedrine/ 

Hydroxyzine 

Theophylline-Iodinated  Glyc¬ 
erol 

Theophylline-Potassium  Iodide 
Triamcinolone  Acetonide 
Zafirlukast 
Zileuton 
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Round-Two  Data  Extraction  Specifications 

Two  steps  were  involved  in  the  final  data  extraction  that  was  con¬ 
ducted  during  round  two.  First,  we  created  the  unique  identifier  for 
all  individuals  identified  as  asthma  patients  using  the  round-one  data. 
Then  we  extracted  data  on  all  encounters  for  the  subset  of  patients 
who  were  continuously  enrolled  at  one  of  the  demonstration  or  con¬ 
trol  MTFs  during  each  of  the  two  study  years. 

Creation  of  the  RAND  ID 

A  single  RAND  ID  was  created  for  each  person  that  corrected  the 
identifiers  for  people  with  incorrect  or  multiple  family  member  prefix 
(FMP)  relationship  variables  in  the  encounter  and  claims  data.  This 
identifier  consisted  of  the  sponsor’s  SSN  plus  a  two-character  rela¬ 
tionship  identifier  selected  based  on  the  data  available  to  us  in  the 
encounter  and  claims  data  we  used  to  identify  our  patient  population. 
For  each  encounter  or  claim  record,  we  identified  the  combination  of 
the  sponsor  SSN  and  the  date  of  birth  of  the  individual  receiving  the 
service,  which  were  the  basis  for  establishing  the  RAND  identifiers. 

The  unique  RAND  ID  was  established  in  a  two-step  process. 
We  first  created  two  files  that  identified  all  DDS  and  FMP  codes  for 
each  unique  SSN/patient  birth  date  combination:  the  first  file  con¬ 
tained  all  of  the  DDS  codes  reported  in  the  network  provider  and 
NMOP  claims  files  for  each  SSN/patient  birth  date  combination,  as 
well  as  counts  of  the  total  records  with  each  DDS.  The  other  file  con¬ 
tained  all  of  the  FMP  codes  in  the  SIDR,  SADR,  and  MTF  phar¬ 
macy  records  for  each  SSN/patient  birth  date  combination,  as  well  as 
counts  of  the  total  records  with  each  FMP.  Then  we  combined  the 
information  from  these  two  files,  generating  a  single  relationship  vari¬ 
able  for  each  sponsor  SSN/patient  birth  date  in  the  encounter  and 
claims  data. 

Combining  the  information  from  these  two  files,  a  single  rela¬ 
tionship  variable  was  generated  for  each  sponsor  SSN/patient  birth 
date  in  the  encounter  and  claims  data.  The  RAND  ID  variable  was 
created  by  combining  the  sponsor  SSN  and  the  new  relationship  vari¬ 
able.  Precedence  was  given  to  the  DDS  code  because  this  is  supposed 
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to  be  consistent  throughout  the  system,  whereas  the  FMP  code  is 
only  consistent  within  facility: 

•  If  there  was  only  one  DDS,  the  relationship  variable  was  set  to 
that  DDS. 

•  Otherwise,  the  variable  was  set  to  the  DDS  with  highest  number 
of  occurrences. 

•  If  there  was  no  DDS,  the  variable  was  set  to  the  FMP  with  high¬ 
est  number  of  occurrences. 

This  coding  process  changed  the  relationship  code  for  an  esti¬ 
mated  1.1  percent  of  the  patient  population.  For  less  than  0.75  per¬ 
cent  of  the  cases,  multiple  SSN/birth  date  combinations  received  the 
same  relationship  variable  assignment.  Most  of  these  were  cases  for 
which  the  date  of  birth  was  entered  incorrectly  (e.g.,  month  and  day 
switched),  which  we  corrected.  Some  were  real  errors  in  assignments, 
which  were  not  corrected. 

When  we  created  the  unique  RAND  ID  variable  to  identify 
patients  correctly  in  the  database,  we  appended  this  variable  to  each 
of  the  encounter  or  claims  data  records  so  that  data  in  the  master  file 
could  be  linked  as  accurately  as  possible  to  service-use  data  for  each 
patient.  Using  these  data,  we  created  a  single  summary  file  that  cap¬ 
tured  most  of  the  relevant  information  from  the  phase  one  encounter 
and  claims  data. 

Creating  a  Unique  Patient  Identifier.  The  first  step  necessary  to 
define  the  study  population  and  establish  a  master  file  for  the  analysis 
was  to  establish  a  unique  identifier  for  each  patient  in  the  database. 
This  was  a  challenging  step  because  the  identifiers  on  the  MTF 
encounter  records  (SIDR,  SADR,  MTF  pharmacy)  were  established 
separately  from  the  identifiers  on  the  network  provider  claims  (inpa¬ 
tient,  outpatient,  NMOP).  As  a  result,  some  of  the  identifiers  on  the 
MTF  records  are  incorrect. 

When  an  eligible  individual  enrolls  in  TRICARE,  he  or  she  is 
assigned  a  unique  identifier  consisting  of  the  SSN  of  the  primary 
beneficiary  (sponsor)  plus  a  suffix  code,  called  the  DDS  code,  that 
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identifies  the  relationship  of  the  individual  to  the  sponsor  (e.g.,  self, 
spouse,  child).  For  example,  multiple  children  or  spouses  are  coded  in 
sequential  order  with  codes  1  through  1 9  designated  for  children  and 
codes  30  through  39  designated  for  spouses.  Because  these  identifiers 
are  assigned  in  the  centralized  TRICARE  data  system,  we  know  that 
each  identifier  is  a  unique  code  that  accurately  identifies  a  person.  All 
network  provider  claims  use  these  identifiers  with  the  DDS  code. 

The  TRICARE  data  system  is  not  accessible  to  the  local  MTEs, 
so  they  use  a  separate  identifier  coding  system  based  on  the  same  logic 
as  the  TRICARE  identifiers,  where  the  identifier  consists  of  the  spon¬ 
sor’s  SSN  plus  a  suffix  code,  called  the  EMP  code,  to  identify  the  rela¬ 
tionship  to  the  sponsor.  The  codes  for  family  member  status  used  by 
the  EMP  code  are  the  same  as  those  for  the  DDS  code.  However, 
because  the  MTEs  do  not  have  access  to  a  central  database  on  the 
family  members,  MTE  personnel  may  code  spouses  or  children  incor¬ 
rectly  because  they  may  not  be  aware  of  previous  spouses  or  older 
children.  As  a  result,  some  individuals  could  be  assigned  someone 
else’s  health-care  encounter  in  our  analysis  if  we  did  not  correct  EMP 
codes  before  constructing  our  analysis  file. 

To  address  this  issue,  we  created  a  unique  RAND  identifier 
using  the  process  described  above.  We  then  created  the  two  files 
described  in  the  process  above  and  appended  the  RAND  ID  to  each 
of  the  encounter  or  claims  data  records. 

Establishing  the  Patient  Master  Files.  We  created  a  master  file 
for  use  in  our  analysis,  each  of  which  contained  a  record  for  each 
asthma  patient  identified.  This  file  contained  one  record  for  each 
asthma  patient  served  by  any  Army  MTE  or  health  center  during  the 
two  study  years.  Each  record  contained  the  patient’s  identifier  vari¬ 
able  created  using  the  RAND  IDs;  summaries  of  the  number  of 
encounter  or  claim  records  of  each  type;  enrollment  summary  vari¬ 
ables;  place  of  service  summary  variables;  and  patient  demographic 
information  of  age,  gender,  and  family  relationship. 

The  study  sample  used  for  the  analysis  of  demonstration  effects 
was  the  subset  of  the  patient  population  who  had  been  continuously 
enrolled  in  TRICARE  Prime  with  a  PCM  at  one  of  the  MTEs 
included  in  either  the  demonstration  or  control  group.  This  smaller 
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Study  sample  was  identified  using  variables  on  their  TRICARE  Prime 
enrollment  history  that  were  derived  using  enrollment  data  from  the 
encounter  and  claims  data.  A  patient  who  was  reported  in  all  encoun¬ 
ter  or  claims  records  as  always  being  enrolled  at  one  of  the  demon¬ 
stration  sites  was  assigned  an  enrollment  code  equal  to  “1,”  and  one 
who  was  reported  as  always  being  enrolled  at  a  control  site  was 
assigned  a  code  equal  to  “2.”  A  separate  variable  on  the  file  identified 
the  MTF  at  which  the  patient  was  enrolled. 

Creation  of  Analysis  Master  Files 

Two  summary  files  containing  patient  demographics  and  other  char¬ 
acteristics  were  created  using  the  phase  one  encounter  data.  One  file 
contains  one  record  per  person  per  study  year;  the  other  file  contains 
one  record  per  person.  Summary  variables  were  derived  by  coding  for 
each  variable  on  individual  encounter  or  claim  records  and  summa¬ 
rizing  them  at  the  person  level  based  on  the  unique  RAND  ID  codes. 
First,  the  following  variables  were  created  for  each  claim  record. 

Study  Year  Study  Year  of  April  1  to  March  31,  where  year  =  0  is 
for  1999-2000  and  year  =  1  for  2000-2001 

Demo-control  Demo  =  any  claim  is  for  service  in  a  demonstration 
MTF  or  indicates  that  the  person  is  enrolled  in  a 
demonstration  site 

Control  =  any  claim  is  for  service  in  a  control  site  or 
indicates  that  the  person  is  enrolled  in  a  control 
site 

Other  =  no  claims  for  service  in  either  a  demonstra¬ 
tion  or  control  site 

Note:  For  network  provider  claims,  this  variable  is 
identified  based  on  the  MTF  catchment  area  vari¬ 
able  or  by  a  Zip  code  matched  to  an  Army  DMIS 
ID 

Active-duty  Identifies  active-duty  personnel  based  on  beneficiary 
category  variable 

Army  Identifies  Army  personnel  based  on  beneficiary  cate¬ 

gory  variable 
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Enrollment  ID 


Identifies  whether  person  enrolled  to  network, 
MTF,  or  not  enrolled  based  on  enrollment  DMIS 


ID 


Then  the  variables  were  summarized  by  person  and  study  year. 
Each  data  source  (e.g.,  SIDR,  SADR,  PEC)  was  summarized  by  per¬ 
son  and  year,  and  data  in  these  summary  files  then  were  combined.  In 
addition  to  the  variables  described  above,  place  of  service  and  enroll¬ 
ment  summary  variables  were  created.  Finally,  patient  demographic 
variables  for  age  and  family  relationship  were  created.  The  variables 
were  defined  using  the  following  coding: 


DMIS  IDs  for  DMIS  IDs  are  the  unique  identifier  codes 
MTF  location  of  assigned  to  each  MTF  or  Army  health  center, 
service  An  array  of  all  DMIS  IDs  for  MTFs  that  were 

locations  of  service  for  the  patient’s  MTF 
encounters  during  the  study  year. 


DMIS  IDs  for  An  array  of  all  DMIS  IDs  for  MTFs  where  the 
TRICARE  patient  was  enrolled,  as  recorded  in  the  patient’s 

Prime  enroll-  MTF  encounters  during  the  study  year, 
ments 


User  population  Takes  one  of  the  following  values  for  location  of 
service  for  the  study  year.  Based  on  the  location 
of  service  DMIS  ID  for  each  MTF  claim. 

Always  Same  =  all  nonmissing  DMIS  IDs  for 
study  year  are  the  same  and,  for  the  second  year, 
also  are  the  same  as  the  last  DMIS  ID  for  the 
fourth  quarter  of  the  previous  year. 

Same  for  SY,  No  Match  with  Previous  SY  =  all 
nonmissing  DMIS  IDs  for  the  study  year  are 
the  same  but  not  the  same  as  last  DMIS  ID  for 
the  fourth  quarter  of  the  previous  year. 

Same  for  SY,  No  DMIS  Previous  SY  =  all  non¬ 
missing  DMIS  IDs  for  the  study  year  are  the 
same  and  no  DMIS  ID  for  the  Fourth  quarter  of 
the  previous  year. 

Not  the  Same  =  nonmissing  DMIS  IDs  are  not 
the  same  for  the  study  year. 

No  Focation  Information  =  any  other  conditions. 
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Enrolled  popula¬ 
tion 


Type  of  TRI¬ 
CARE  Prime 
enrollment 


Takes  one  of  the  following  values  for  enrollment 
information  for  the  study  year.  Based  on  the 
enrollment  DMIS  ID  for  each  MTE  or  HCSR 
claim. 

Always  Same  =  all  nonmissing  enrollment  DMIS 
IDs  for  study  year  are  the  same  and,  for  the  sec¬ 
ond  year,  also  are  the  same  as  the  last  DMIS  ID 
for  the  fourth  quarter  of  the  previous  year. 

Same  for  SY,  No  Match  with  Previous  SY  =  all 
nonmissing  enrollment  DMIS  IDs  for  the  study 
year  are  the  same  hut  not  the  same  as  the  last 
DMIS  ID  for  fourth  quarter  of  the  previous 
study  year. 

Same  for  SY,  No  DMIS  Previous  SY  =  all  non¬ 
missing  enrollment  DMIS  IDs  for  the  study 
year  are  the  same  and  no  DMIS  ID  for  the 
fourth  quarter  of  the  previous  year. 

Enrolled,  Not  the  Same  =  nonmissing  enrollment 
DMIS  IDs  exist  for  the  study  year  hut  are  not 
the  same. 

PEC  Claims  Only  for  Year  =  MTE  pharmacy 
records  exist  hut  no  enrollment  DMIS  IDs  on 
service-use  records  for  the  study  year. 

Never  Enrolled  in  Year  =  neither  any  enrollment 
DMIS  IDs  on  service-use  records  nor  any  MTE 
pharmacy  records  for  the  study  year. 

Takes  one  of  the  following  values  for  type  of 
enrollment  for  the  study  year: 

Enrolled  to  Demo  =  enrollment  DMIS  ID  is  ever 
a  demo  site  in  study  year. 

Enrolled  to  Control  =  enrollment  DMIS  ID  is 
ever  a  control  site  in  study  year. 

Enrolled  to  Network:  Demo  =  enrollment  DMIS 
ID  is  ever  network,  patient  Zip  code  in  demo 
catchment  area,  and  place  of  service  DMIS  ID 
is  ever  a  demo  site. 

Enrolled  to  Network:  Control  =  enrollment 
DMIS  ID  is  ever  network,  patient  Zip  code  in 
control  catchment  area,  and  place  of  service 
DMIS  ID  is  ever  a  control  site. 

Enrolled  to  Network:  Other  =  last  enrollment 
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DMIS  ID  for  study  year  is  network  in  demo  or 
control  catchment  area  and  place  of  service 
DMIS  ID  is  never  a  demo  or  control  site. 

Enrolled  to  Other  MTF  =  last  enrollment  DMIS 
ID  for  study  year  is  not  missing  and  is  not 
within  networks  of  interest. 

Never  Enrolled  in  Year  =  enrollment  DMIS  ID  is 
missing  for  entire  study  year  and  no  MTF 
pharmacy  claims. 

PEC  Claims  Only  for  Year  =  MTF  pharmacy 
records  exist  hut  no  enrollment  DMIS  IDs  on 
service-use  records  for  the  study  year. 

Analysis  popula-  Indicates  whether  person  is  in  the  analysis  popu- 
tion  lation  for  the  study  year.  Based  on  enrolled 

population  (enrpop). 

1  =  If  “enrolled  population”  variable  is  “Always 
Same”  or  “Same  for  SY,  No  DMIS  prev  SY” 
and  the  first  enrollment  DMIS  ID  is  a  demon¬ 
stration  site. 

2  =  if  “enrolled  population”  variable  is  “Always 
the  Same”  or  “Same  for  SY,  No  DMIS  prev  SY” 
and  the  first  enrollment  DMIS  ID  is  a  control 
site. 

0  =  Otherwise. 


Age  for  study  year 

Five-year  age 
categories 

Relationship  cate¬ 
gory 


Age  at  December  3 1 ,  calculated  from  date  of 
birth  and  study  year. 

Four-level  age  category  variable  for  study  year: 

Eess  than  18  years,  18-44  years,  45-64  years,  or 
65+  years. 

Classified  as  child,  spouse,  parent,  or  other. 


Creation  of  Identifier  Files  for  Data  Extraction  from  DDS  and  FMP 
Files 

For  the  round-two  data  extraction  process,  two  files  were  sent  to 
PASBA  that  contained  lists  of  unique  patient  identifiers  for  all  the 
asthma  patients  we  identified  as  being  in  our  sample  for  the  analysis 
of  guideline  effects.  We  requested  an  extraction  of  all  encounter 
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records  for  these  patients  for  all  MTF  and  network  provider  services 
and  pharmacy  prescriptions.  One  file  was  to  he  used  to  extract  all 
claims  for  each  patient  identifier  from  files  that  use  the  DDS 
relationship  variable  (outpatient  and  inpatient  network  provider  files, 
NMOP  files),  and  the  other  was  to  he  used  to  extract  claims  from 
files  that  use  the  FMP  identifier  (SIDR,  SADR,  PEC  pharmacy  data). 

To  create  the  identifier  files,  an  index  file  was  constructed  that 
contained  all  unique  sponsor  SSN/FMP  and  sponsor  SSN/DDS 
combinations  for  all  patients  identified  as  continuously  enrolled  at 
one  of  the  demonstration  or  control  sites.  All  identifiers  reported  in 
all  the  MTF  encounter  records  and  network  provider  HCSRs  were 
captured.  Patient  identifier  records  then  were  output  to  each  of  the 
two-identifier  files,  using  the  following  rules: 

A.  Identifier  file  for  extraction  of  data  from  files  using  the  DDS  rela¬ 
tionship  variable  (network  provider  NCSRs,  NMOP) — 

1.  If  the  relationship  variable  is  based  on  a  DDS  then  output  the 

record  with  sponsor  SSN  and  DDS. 

2.  If  the  relationship  variable  is  based  on  FMP: 

a.  If  the  person  is  a  child  (FMP  1-19)  then  do  the  following: 

1)  If  FMP  is  1-4  then  output  one  record  for  each  FMP  1-4 
(four  records) 

2)  If  FMP  is  5-19  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  1-4  (five  records). 

b.  If  the  person  is  a  spouse  (FMP  30-39): 

1)  If  FMP  is  30  or  31  then  output  one  record  for  both  FMP 
30  and  31  (two  records) 

2)  If  FMP  is  32-39  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  30  and  31  (three  records). 

c.  If  the  person  has  any  other  code,  output  one  record. 

B.  For  identifier  files  for  extraction  of  data  from  files  using  the  FMP 
relationship  variable  (SIDR,  SADR,  MTF  pharmacy)  only  the  FMP 
variable  is  used  because  there  is  no  DDS  identifier  in  the  encounter 
records — 
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a.  If  the  person  is  a  child  (FMP  1-19)  then  do  the  following: 

1)  If  FMP  is  1-4  then  output  one  record  for  each  FMP  1-4 
(four  records) 

2)  If  FMP  is  5-19  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  1-4  (five  records). 

h.  If  the  person  is  a  spouse  (FMP  30-39)  then  do  the  following: 

1)  If  FMP  is  30  or  31  then  output  one  record  for  hoth  FMP 
30  and  31  (two  records) 

2)  If  FMP  is  32-39  then  output  record  with  this  FMP  as  well 
as  one  record  for  each  FMP  30  and  31  (three  records). 

c.  If  the  person  has  any  other  code,  output  one  record. 

Definition  of  Key  Outcome  Variables 

Indicators  selected  for  the  evaluation  of  guideline  effects  were  those 
that  could  he  measured  using  available  administrative  data  on  health¬ 
care  encounters,  use  of  prescription  medications,  and  the  demo¬ 
graphic  and  clinical  characteristics  of  the  patients.  The  data  employed 
to  derive  these  variables  were  in  the  master  file  for  the  asthma  patients 
in  the  study  sample,  along  with  the  comprehensive  encounter  data 
obtained  in  the  round-two  data  extraction.  For  each  outcome  vari¬ 
able,  we  subset  the  records  for  the  sample  from  the  master  file  and 
used  the  unique  RAND  ID  codes  to  merge  patient  data  to  the  round- 
two  encounter  or  claims  data  required  to  derive  the  variable.  When 
we  obtained  round-two  data  from  PASBA,  we  appended  the  RAND 
ID  codes  to  each  encounter  or  claim  record  in  the  data  using  the 
sponsor  SSN/patient  birth  date  combination  as  the  linking  variable. 

We  could  obtain  the  data  for  all  encounters  from  the  round-two 
data  extraction  except  the  network  provider  outpatient  services.  The 
DoD  files  containing  these  records  were  extremely  large  because  they 
contained  records  for  all  TRICARE  beneficiaries.  Because  of  limita¬ 
tions  of  the  data  system  on  which  PASBA  operated  to  extract  the 
data,  they  were  not  able  to  use  standard  data  management  methods  to 
merge  the  index  file  of  patient  identifiers  to  the  identifiers  on  the 
claims  data.  They  attempted  to  perform  the  data  abstraction  on  sub¬ 
sets  of  DoD  data  by  month  of  year,  but  the  resulting  data  files  were 
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inconsistent  in  format  and  content,  and  we  were  not  able  to  use 
them. 

The  specific  data  elements  are  defined  below,  followed  by  a  table 
of  the  codes  used  to  measure  them: 

Enrollment  Type.  Two  variables  were  derived  and  used  to  define 
enrollment  status  at  a  demonstration  or  control  MTF.  The  first  was  a 
variable  that  identified  whether  a  patient  was  ever  enrolled  at  one  of 
these  MTFs  or  enrolled  with  a  network  provider  but  resided  in  the 
catchment  area  of  one  of  the  MTFs.  The  second  variable  identified 
whether  the  patient  was  continuously  enrolled  in  the  same  place  or 
changed  enrollment  during  the  relevant  year.  Using  these  two  vari¬ 
ables,  we  included  in  our  sample  any  patient  who  was  continuously 
enrolled  in  the  same  place  and  that  place  was  one  of  the  demonstra¬ 
tion  or  control  MTFs.  The  enrollment  type  variables  also  were  used 
to  identify  comparison  populations  for  some  of  the  analyses. 

Patient  Age.  The  variable  for  patient  age  was  defined  as  the  age 
of  each  patient  at  the  end  of  each  calendar  year,  which  was  calculated 
using  the  date  of  birth  variable  established  on  the  master  file.  We  also 
derived  a  categorical  variable  for  patient  age,  which  was  used  in  some 
of  the  analyses.  This  five-level  variable  had  the  age  categories  of  <6 
years,  6-18  years,  18-44  years,  45-64  years,  and  65  years  and  older. 

Asthma  Control  Medication  Use.  Three  categories  of  asthma 
control  medications  were  defined,  reflecting  different  aspects  of 
asthma  management.  Long-term  controller  medications  are  taken  to 
prevent  occurrence  of  exacerbations,  and  complementary  medications 
assist  in  the  control  process.  Using  the  National  Drug  Category  codes 
in  the  MTF  pharmacy  data  and  NMOP  data,  long-term  controllers 
were  defined  to  include  inhaled  corticosteroid,  leukotriene  inhibitor, 
Beta2-agonist/CS,  and  oral  corticosteroid.  Complementary  medica¬ 
tions  were  defined  to  include  anticholinergic,  Beta2-agonist/LA,  and 
methylxanthine.  Short-acting  rescue  medication  was  defined  as  Beta2- 
agonist.  For  each  medication  group,  patients  with  at  least  one  pre¬ 
scription  for  the  medication  during  a  study  year  in  either  the  MTF 
pharmacy  or  NMOP  data  were  coded  as  using  the  medication. 

Use  of  Outpatient  Services.  For  each  patient  in  the  study  sam¬ 
ple,  asthma-related  outpatient  visits  were  identified  from  the  data  in 
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the  SADR  because  these  patients  were  getting  their  outpatient  care  at 
the  MTF  at  which  they  were  enrolled.  An  asthma-related  outpatient 
visit  was  defined  as  an  MTF  outpatient  clinic  encounter  with  an  asth¬ 
ma  diagnosis  code,  excluding  emergency  room  visits.  The  asthma- 
related  outpatient  visits  were  summed  for  each  patient,  and  visit  rates 
were  calculated  as  the  sum  of  visits  across  all  patients  in  a  group 
divided  by  the  number  of  these  patients. 

Use  of  Emergency  Room  Services.  We  were  able  to  measure 
asthma-related  emergency  room  visits  only  for  MTF  emergency 
rooms  because  we  did  not  have  the  data  needed  to  identify  emergency 
room  visits  in  the  network  provider  outpatient  data.'  The  emergency 
room  visits  probably  are  undercounted  because  of  these  missing  data, 
but  we  believe  that  the  undercount  is  small  because  most  patients 
enrolled  at  an  MTF  are  likely  to  use  the  MTF  emergency  room  when 
they  need  such  care.  An  asthma-related  emergency  room  visit  was 
defined  as  an  emergency  room  encounter  with  an  asthma  diagnostic 
code.  The  emergency  room  visits  were  summed  for  each  patient,  and 
visit  rates  were  calculated  as  the  sum  of  visits  across  all  patients  in  a 
group  divided  by  the  number  of  these  patients. 

Hospital  Inpatient  Use.  A  variable  was  created  that  contained  the 
count  of  asthma-related  hospital  inpatient  stays  for  each  patient  in 
the  sample,  including  stays  at  MTFs  and  community  hospitals  that 
are  network  providers.  An  asthma-related  inpatient  stay  was  defined 
as  an  inpatient  encounter  with  an  asthma  diagnostic  code.  The  inpa¬ 
tient  stays  were  summed  for  each  patient,  and  hospitalization  rates 
were  calculated  as  the  sum  of  all  inpatient  stays  across  the  patients  in 
the  sample  divided  by  the  number  of  patients. 


'  There  is  a  variable  in  the  HCSRs  that  identifies  the  type  of  outpatient  encounter,  so  it  is 
possible  to  identify  network  provider  emergency  room  visits.  However,  we  had  not  obtained 
that  variable  in  the  network  provider  data  for  this  study,  so  we  had  to  work  with  only  the 
MTF  emergency  room  information. 
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Cost  Estimation  Methodology 

Estimation  of  MTF  Unit  Costs 

To  estimate  the  costs  of  care  for  asthma  patients  at  the  Army  MTFs 
included  in  this  study,  MEPRS  financial  data  were  used  to  develop 
sets  of  unit  costs  for  inpatient  and  outpatient  encounters.  The  rele¬ 
vant  estimated  unit  cost  then  was  applied  to  each  unit  of  service 
included  in  the  SIDR  and  SADR  encounter  records.  We  note  there 
has  been  some  criticism  in  DoD  that  the  MEPRS  data  overestimates 
the  MTEs’  costs  of  doing  business.  The  source  of  this  criticism  is  a 
reported  overestimation  of  the  available  hours  of  military  personnel 
time  for  patient  care  activities  because  personnel  often  do  not  record 
time  that  they  spend  on  military-related  activities.  While  acknowl¬ 
edging  this  issue,  we  also  understand  that  MEPRS  offers  the  best 
available  data,  and  it  is  the  basis  for  all  other  cost  estimations  for  the 
demonstration. 


Indicator  of  Guide¬ 
line  Effect 

Codes  Used  for  the  Definition 

Percentage  of 
asthma  patients 
using  long-term 
controllers 

Use  of  long-term  controller  medication  was  defined  as  hav¬ 
ing  at  least  one  prescription  for  an  inhaled  corticosteroid, 
leukotriene  inhibitor,  Beta2  agonist/CS,  or  oral. 

Percentage  of 
asthma  patients 
using  complemen¬ 
tary  medications 

Use  of  complementary  medications  was  defined  as  having  at 
least  one  prescription  for  Beta2  agonist/LA  or  methyl- 
xathine. 

Percentage  of 
asthma  patients 
using  short-acting 
rescuer  medications 

Use  of  short-acting  rescuer  medications  was  defined  as  hav¬ 
ing  at  least  one  prescription  for  Beta2  agonist. 

Number  of  outpa¬ 
tient  visits  per 
asthma  patients 

Outpatient  visit  was  defined  as  a  visit  to  any  MTF  clinic 
except  the  emergency  room  (MEPRS  code  Bl). 

Number  of  emer¬ 
gency  room  visits 
per  asthma  patient 

MTF  emergency  room  visit  was  identified  using  MEPRS  code 

Bl  (network  provider  emergency  room  use  data  were  not 
available  for  this  study,  but  also  should  be  included). 

Number  of  hospitali¬ 
zations  per  asthma 
patient 

Each  SIDR  or  network  provider  inpatient  encounter  was 
identified  as  a  hospitalization. 
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The  cost  estimation  methodology  we  developed  mirrors  its 
approach  to  the  PLCA  method  developed  hy  SRA  International  for 
the  Medicare-DoD  Subvention  Demonstration.  For  this  cost  analysis, 
we  used  cost  and  workload  data  that  SRA  generated  for  MTF  outpa¬ 
tient  clinics  or  inpatient  wards  for  all  MTFs  in  the  DoD  system  for 
fiscal  year  1998.  The  estimated  unit  costs  included  total  direct  and 
indirect  expenses  for  each  MTF  cost  center  (ward  or  clinic) ,  including 
direct  expenses  for  staff  time  and  supplies  as  well  as  indirect  expenses 
for  ancillary  clinical  services,  administrative  services,  and  maintenance 
and  other  support  services.  We  summarize  here  the  methodology  for 
calculating  the  inpatient  and  outpatient  costs. 

We  updated  the  unit  costs  to  FY  1999  estimates  hy  applying  an 
inflation  factor  of  1.4  percent.  These  same  unit  costs  were  applied  to 
encounters  for  both  study  years.  By  holding  costs  constant  over  time, 
any  observed  changes  in  costs  between  study  years  one  and  two  can 
be  attributed  to  changes  in  utilization. 

We  tested  two  references  for  Medicare  cost  increases  to  deter¬ 
mine  the  1.4  percent  inflation  rate.  The  first  was  the  trend  in  the 
U.S.  per-capita  costs  (USPCC)  for  fee-for-service  beneficiaries  that 
the  Health-Care  Financing  Administration  Office  of  the  Actuary 
calculates  each  year.  For  the  years  1996  through  1999,  the  USPCC 
increased  at  an  annual  rate  of  1.4  percent.  We  also  used  the  annual 
rate  of  increase  in  the  M+C  county-level  capitation  rates,  which  the 
Balanced  Budget  Act  of  1 997  mandated  are  to  be  equal  to  the  rate  of 
increase  in  Medicare  fee-for-service  costs.  The  annual  updates  used  by 
HCFA  to  establish  the  capitation  rates  for  calendar  years  1999  and 
2000  were  1.88  percent  and  0.90  percent  increases,  respectively,  over 
the  previous  year.  These  also  average  to  1.4  percent.  Because  DoD 
payment  policies  mirror  Medicare  policies,  payments  discounted 
using  this  inflation  rate  represent  increases  in  what  either  DoD  or 
Medicare  would  have  paid  community  providers  if  the  service  had 
been  provided  in  FY  1998  instead  of  FY  1999. 

Inpatient  Stays 

We  estimated  the  cost  per  inpatient  stay  for  each  MTF  inpatient  stay 
using  the  following  formula: 
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Cost  for  inpatient  stay  i  in  wardy  = 

(medical  per  diem  cost)^^  x  (number  of  days)^y  + 

(surgical  per  diem  cost),y  x  (number  of  days),^  + 
surgical  cost  for  surgical  DRG, 

where  the  number  of  bed  days  for  each  type  of  inpatient  ward — 
medical  or  surgical — is  the  sum  of  the  ward  and  ICU  days  in  the 
SIDR.  DRG  is  the  Diagnosis-Related  Group  assigned  to  each  inpa¬ 
tient  stay  based  on  the  patient’s  principal  diagnosis  and  treatment. 
Medicare  uses  DRGs  as  the  basis  for  payments  for  inpatient  services, 
and  DoD  uses  DRGs  to  establish  amounts  billed  to  third-party  insur¬ 
ers  for  MTF  inpatient  services. 

•  For  each  inpatient  ward  in  an  MTF  identihed  by  the  MEPRS 
level-3  accounts  (the  level  that  inpatient  wards  are  coded  in  the 
SIDR  data),  we  obtained  the  following  MEPRS  data  that  we 
used  to  calculate  average  total  per  diem  expenses:  Total  expenses 
including  all  stepped-down  expenses  from  MEPRS  accounts  D 
and  E  except  for  surgical  expenses  (anesthesia,  surgery  suite,  and 
recovery  room  expenses). ^  These  costs  included  clinical  salaries, 
direct  operating  costs,  support  costs,  allocated  intensive-care 
unit  and  ancillary  service  costs,  allocated  costs  from  purification 
of  cost  pools  that  contain  costs  related  to  more  than  one 
account,  and  resource  sharing  costs  that  SRA  assigned  to  the 
inpatient  ward. 

•  Total  number  of  occupied  bed  days  (OBD)  during  the  year, 
which  will  be  used  with  total  expenses  to  generate  an  estimated 
total  expense  per  OBD. 

•  Eor  each  surgical  DRG,  we  obtained  an  estimated  average  MTE- 
level  surgical  expense  that  included  expenses  for  anesthesia,  sur¬ 
gery  suite,  and  recovery  room.  This  cost  estimate  was  derived  as 


^  The  MEPRS  D  accounts  are  clinical  ancillary  services  (e.g.,  pharmacy,  pathology,  intensive 
care),  and  the  E  accounts  are  support  services  (e.g.,  administration,  housekeeping,  laundry, 
depreciation). 
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the  total  MTF  surgical  expenses  divided  by  the  total  weights  of 
surgical  DRGs  during  the  year,  where  surgical  costs  were  esti¬ 
mated  using  the  same  method  that  SRA  applied  for  the  PLCA 
calculations.  For  each  surgical  disposition,  we  multiplied  the 
MTF  average  surgical  cost  by  the  DRG  weight  for  the  DRG 
assigned  to  the  patient  stay. 

This  approach  allowed  us  to  capture  all  expenses  for  an  inpatient 
stay  using  a  consistent  methodology  across  all  the  years  of  inpatient 
records  included  in  our  analysis.  This  method  smooths  out  errors  in 
reporting  movement  of  patients  between  IGUs  and  regular  inpatient 
wards  by  estimating  average  per-diem  costs  that  include  costs  for  the 
regular  ward  services  plus  related  IGU  services.  At  the  same  time,  it 
captures  the  onetime  costs  associated  with  the  surgical  procedure  per¬ 
formed  for  each  surgical  stay  by  applying  these  costs  separately  for 
each  event.  The  method  also  allows  costs  to  increase  with  length  of 
stay,  thereby  capturing  some  of  the  additional  costs  incurred  by  the 
older  population.  However,  this  approach  assumes  that  ancillary  costs 
are  a  linear  function  of  days,  whereas  it  is  known  that  these  costs  tend 
to  be  concentrated  in  the  early  days  of  an  inpatient  stay  (Garter  and 
Melnick,  1990).  Therefore,  the  method  sacrifices  some  precision  in 
estimating  ancillary  service  costs,  although  SRA  has  informed  us  that 
total  MTF  ancillary  costs  correlate  strongly  with  length  of  stay. 

Outpatient  Visits 

For  each  clinic  in  an  MTF  identified  by  the  MEPRS  level-4  accounts 
(the  level  that  clinics  are  coded  in  the  SADR  data),  we  obtained  the 
following  MEPRS  data  that  we  used  to  calculate  average  total 
expenses  per  outpatient  visit: 

•  Total  MEPRS  level-4  expenses  for  the  clinic  for  each  year, 
including  the  resource-sharing  expenses  that  SRA  has  estimated 
and  assigned  to  each  clinic. 

•  The  MEPRS  count  of  total  outpatient  visits  in  the  clinic  during 
the  year. 
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•  Within  the  total  expenses,  separate  identification  of  the  expenses 
for  laboratory,  radiology,  pharmacy,  all  other  ancillary  services 
(including  allocated  costs  from  purification  of  cost  pools),  and 
resource  sharing. 

These  data  allowed  us  to  calculate  the  average  total  cost  per  visit 
for  each  clinic  in  an  MTF  and  to  estimate  the  shares  of  the  total  clinic 
expenses  that  are  attrihutahle  to  ancillary  services. 


APPENDIX  C 

Modules  of  the  Climate  Survey 
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GUIDELINE  IMPLEMENTATION  SURVEY 
RAND  Process  Evaluation 
(Team  Level) 

What  is  the  name  of  your  MTF: _ 


Completion  of  this  survey  is  voluntary.  Y ou  may  skip  any  question  that  you  do  not  want  to 
answer.  Please  understand  that  your  answers  are  completely  private  and  confidential.  Your 
identity  will  never  be  attached  to  the  opinions  and  experiences  expressed  in  this  survey. 


Please  feel  free  to  use  the  back  of  this  survey  booklet  to  give  us  your  reactions  to  our  process 
evaluation,  tell  us  about  your  experience  at  the  conference,  or  anything  else  you  think  is 
important.  THANK  YOU  VERY  MUCH  FOR  YOUR  TIME  AND  PARTICIPATION. 
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MODULE  A  THROUGH  C 

Dimensions  of  motivation  addressed  in  Module  A: 

•  Improve  quality  of  care  for  patients 

•  Improve  patient  satisfaction  with  their  care 

•  Improve  the  efficiency  of  patient  care 

•  Reduce  error  in  treatment,  ordering  tests,  and  medication 

•  Improve  decisions  for  specialty  referrals 

•  Make  the  joh  easier 

•  Increase  the  satisfaction  with  what  is  being  accomplished 

•  Reduce  legal  liahility  exposure. 

Module  B  Items  Module  C  Items 

Supportiveness  of  Climate  for  Guidelines  Attitudes  About  Practice  Guidelines 

(four-level  response  from  "no  action"  to  (Scale  of  1  =  strongly  disagree  to  7  = 

"strong  action")  strongly  agree) 

How  likely  is  it  that  a  staff  member  in  your  Practice  guidelines  (do  not)  over- 
MTF  would  be  noticed  if  he  or  she  did  not  simplify  diagnostic  and  treatment 
cooperate  with  guideline  implementation?  decisions  in  medicine. 

How  risky  would  it  be  for  a  staff  member  in  Practice  guidelines  could  help  me 
your  MTF  not  to  cooperate  with  guideline  deliver  better  patient  care, 
implementation? 

What  do  you  think  would  be  done  if  man-  Use  of  practice  guidelines  in  medi- 
agement  noticed  that  a  staff  member  was  cine  will  (not)  limit  a  physician's 
cooperating  with  guideline  freedom  to  take  action, 

implementation? 

What  do  you  think  would  be  done  if  man-  Practice  guidelines  help  reduce 
agement  noticed  that  a  staff  member  was  variation  in  clinical  practice, 
not  cooperating  with  guideline  implemen¬ 
tation? 

How  likely  is  it  that  management  would  Use  of  practice  guidelines  will  (not) 

encourage  a  staff  member  to  follow  proce-  reduce  provider  efficiency, 
dures  established  to  implement  the  guide¬ 
line? 

How  likely  is  it  that  management  would  Use  of  practice  guidelines  is  a  good 

praise  a  staff  member  for  cooperating  with  way  to  summarize  and  reinforce 
guideline  implementation?  scientific  evidence  on  diagnosis 

and  management  of  specific  con¬ 
ditions. 

How  likely  is  it  that  management  would 
notice  that  a  staff  member  did  not  have  the 
resources  to  follow  guideline  procedures? 
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MODULE  E 


INSTRUCTIONS 

In  this  section  you  are  asked  to  assess  your  MTF's  efforts  to  improve  the  quality  of  care  and 
services  it  provides.  Please  read  each  statement  carefully.  Indicate  the  extent  to  which  you  agree 
or  disagree  that  the  statement  characterizes  your  MTF  by  circling  the  appropriate  response  ( 1  = 
Strongly  Disagree,  5  =  Strongly  Agree).  In  answering  the  questions,  you  should  think  about 
what  the  MTF  is  actually  like  now,  not  how  you  think  it  might  be  in  the  future  or  how  you  might 
wish  it  to  be. 


RESPONSE  CATEGORIES 

In  circling  a  response,  please  keep  in  mind  the  following  general  guidelines  regarding  the 
ehoiees  of  response  categories: 

•  Circle  Strongly  Agree  when  the  statement  represents  a  completely  accurate 
description  of  your  MTF, 

•  Circle  Strongly  Disagree  when  the  description  is  completely  inaccurate. 

•  Circle  Neither  Agree  Nor  Disagree  when  you  believe  the  statement  is  neither  a 
particularly  accurate  nor  a  particularly  inaccurate  description  of  your  M'fF.  This 
situation  may  arise  because  there  is  wide  variation  in  the  activities  the  statement 
describes.  For  example,  you  might  circle  neither  agree  nor  disagree  when  the 
statement  is  true  of  some  departments  but  not  of  others. 

•  Circle  “Don’t  Know”  if  you  do  not  have  enough  information  to  answer  a  question. 


GLOSSARY/SPECIAL  INSTRUCTIONS 


MTF: 

When  asked  to  make  a  global  judgement  about  your  MTF,  please 
respond  based  upon  your  knowledge  and  experienee  of  the  department 
or  area  in  which  you  are  currently  employed,  the  other  departments  you 
come  in  contact  with,  and  the  information  you  have  on  your  MTF  as  a 
whole. 

Quality  of  Care 
and  Services: 

Throughout  the  survey  you  are  asked  to  make  judgements  about  the 
“quality  of  care  and  services  provided.”  "Quality  of  care  and  services” 
is  a  general  category  and  refers  to  the  technical  quality  of  care  to 
patients  and  how  well  patient  services  needs  are  met  in  your  MTF. 
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Slronglv  Disagree 
Disagree 

LEADERSHIP 

1 .  The  Command  Group  provides  highly 

visible  leadership  in  maintaining  an  1  2 

environment  that  supports  quality 

improvement. 

2.  The  Command  Group  consistently 

participates  in  activities  to  improve  the  1  2 

quality  of  care  and  services. 

3.  The  Command  Group  has  demonstrated 

an  ability  to  manage  the  changes  (e.g..  I  2 

organizational,  technological)  needed  to 
improve  the  quality  of  care  and  services. 

4.  The  Command  Group  acts  on  suggestions  1  2 

to  improve  the  quality  of  care  and 

services. 

5.  The  Command  Group  generates  1  2 

confidence  that  efforts  to  improve  quality 

will  succeed. 

INFORMATION  AND  ANALYSIS 

6.  The  MTF  uses  a  wide  range  of  data  and 

information  about  the  quality  of  care  and  I  2 

services  to  make  improvements. 

7.  The  MTF  continually  tries  to  improve 

how  it  uses  data  and  information  on  the  1  2 

quality  of  care  and  services. 

8.  The  MTF  continually  tries  to  improve  the 

accuracy  and  relevance  of  its  data  on  the  I  2 

quality  of  care  and  services  provided. 

9.  The  MTF  continually  tries  to  improve  the 

timeliness  of  its  data  on  the  quality  of  1  2 

care  and  services  provided. 

EMPLOYEE  INVOLVEMENT  IN 
QUALITY  PLANNING 

10.  MTF  staff  are  involved  in  developing  1  2 

plans  for  improving  quality. 


Neither 

Agree 

Don't 
Know 


Nor  Agree  Strongly 
Disagree  Agree 


3  4  5  9 


3  4  5  9 

3  4  5  9 

3  4  5  9 

3  4  5  9 


3  4  5  9 


3  4  5  9 

3  4  5  9 

3  4  5  9 


3  4  5  9 
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Neilher 


11.  Non-managerial  staff  are  playing  a  key  I  2  3  4  5  9 

role  in  setting  priorities  for  quality 

improvement. 

HIJV1.\N  RESOI  RCE  llTILIZ.\TION 

12.  MTF  staff  are  given  education  and 

training  in  how  to  identify-  and  act  on  I  2  3  4  5  9 

quality  improvement  opportunities. 

1 3.  MTF  staff  are  given  education  and 

training  in  statistical  and  other  I  2  3  4  5  9 

quantitative  methods  that  support  quality 

improvement. 

14.  MTF  staff  are  given  the  needed  education 

and  training  to  improve  job  skills  and  I  2  3  4  5  9 

performance. 

1 5.  MTF  staff  are  rew  arded  and  recognized 

(e.g.,  financially  and/or  otherwise)  for  I  2  3  4  5  9 

improving  quality. 

1 6.  MTF  staff  have  the  authority  to  correct 

problems  in  their  area  when  quality  I  2  3  4  5  9 

standards  are  not  being  met. 

17.  MTF  staff  are  supported  when  they  take  I  2  3  4  5  9 

necessary  risks  to  improve  quality. 

1 8.  The  MTF  has  an  effective  system  for 

employees  to  make  suggestions  to  I  2  3  4  5  9 

management  on  how  to  improve  quality. 

QIIALTI Y  MANAGEMENT 

19.  The  quality  assurance  staff  effectively 

coordinate  their  efforts  with  others  to  I  2  3  4  5  9 

improve  the  quality  of  care  and  services 
the  hospital  provides. 

20.  The  MTF  has  effective  policies  to  I  2  3  4  5  9 

support  improving  the  quality  of  care  and 

services. 

21.  The  MTF  works  closely  with  suppliers  to 

improve  the  quality  of  their  products  and  I  2  3  4  5  9 

services. 


ol 
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Neither 

Agree 

Strongly  Disagree  Nor  Agree  Strongly  Don't 
Disagree  Disagree  Agree  Know 


22.  The  MTF  tries  to  design  quality  into  new  I 
services  as  they  are  being  developed. 

23.  The  NfTF  views  quality  assurance  as  a  I 

continuing  search  for  ways  to  improve. 

QUALITY  RESULTS 

24.  The  MTF  has  done  a  good  job  of  I 

simplifying  how  care  and  services  are 
provided. 

25.  Over  the  past  few  years,  the  MTF  has 

shown  steady,  measurable  improvements  I 

in  the  quality  of  care  provided  to  medical, 
surgical  and  obstetric  patients. 

26.  Over  the  pa.st  few  years,  the  MTF  has 

shown  steady,  measurable  improvements  I 

in  the  quality  of  care  provided  by  clinical 
support  departments  such  as  laboratory, 
pharmacy,  and  radiology. 

27.  Over  the  past  few  years,  the  MTF  has 

shown  steady,  measurable  cost  reduction  I 

while  maintaining  or  Improving  quality. 

CUSTOMER  SATISFACTION 

28.  The  MTF  does  a  good  job  of  assessing  I 

current  patient  needs  and  expectations. 

29.  MTF  staff  promptly  resolve  patient  I 

complaints. 

30.  Patients'  complaints  are  studied  to 

identify  patterns  and  prevent  the  same  I 

problems  from  recurring. 

3 1 .  The  MTF  uses  data  from  patients  to  I 

improve  services. 

32.  The  MTF  uses  data  on  customer 

expectations  and/or  satisfaction  when  I 

designing  new  services. 


2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 


2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 

2  3  4  5  9 
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Asthma  Guideline  Implementation  at  Nine  Months 

Primary  Care  Providers  Questionnaire 

This  questionnaire  is  designed  to  identify  areas  of  the  asthma  guide¬ 
line  and  toolkit  items  that  may  need  further  attention  and  to  help 
support  implementation  of  clinical  practice  guidelines.  It  is  also 
designed  to  assess  the  range  of  effects  (both  positive  and  negative)  the 
use  of  the  guideline  is  having  on  Army  medical  staff,  clinical  practice, 
and  patient  care. 

Completion  of  this  questionnaire  is  voluntary.  You  may  skip  any 
question  that  you  do  not  want  to  answer.  Please  understand  that  your 
answers  are  completely  private  and  confidential.  Your  identity  will 
never  he  attached  to  the  opinions  and  experiences  expressed  in  this 
survey. 

Thank  you  very  much  for  your  time  and  participation. 
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SECTION  A:  ABOUT  THE  ASTHMA  GUIDELINE 


Card  1 
FORM  3 

1.  Have  you  received  a  copy  of  the: 


(Check  One  Box  o 

n  Each  Line) 

Yes 

No 

a.  Full  asthma  guideline? 

□  ; 

□2 

3/ 

b.  Laminated  one-page  asthma 
guideline  containing  its  key 
elements 

□  ; 

□2 

47 

2.  Do  you  agree  or  disagree  with  the  following  statements  about  the  DoD/VA  asthma  guideline? 

(Check  One  Box  on  Each  Line) 

The  DoD/VA  Asthma  guideline . . . 


Strongly 

Apree 

Apree 

Neither  Agree 
nor  Disapree 

Disapree 

Strongly 

Disapree 

a. 

has  helped  me  provide  better  care  to  my 
asthma  patients 

□  ; 

□2 

□2 

□4 

□5 

5/ 

b. 

has  no  effect  on  the  way  1  treat  my 
asthma  patients 

□  ( 

□2 

□2 

□4 

□5 

6/ 

c. 

is  confusing  for  me  to  follow 

□  ; 

□2 

□2 

□4 

□5 

7/ 

d 

Is  applicable  to  all  asthma  patients 

□  ; 

□2 

□2 

□4 

□5 

8/ 

e. 

has  reduced  my  flexibility  to  treat  asthma 
patients 

□  r 

□2 

□2 

□4 

□s 

9/ 

f 

has  increased  the  time  1  spend  with 
asthma  patients 

□  ( 

□2 

□2 

□4 

□s 

10/ 

9- 

has  made  it  easier  to  communicate 
treatment  plans  to  patients 

□  r 

□2 

□2 

□4 

□s 

11/ 

h. 

has  required  me  to  see  my 
asthma  patients  more  frequently 

□  ( 

□2 

□2 

□4 

□s 

12/ 

1. 

has  taught  me  something  1  did  not  know 
before 

□  ; 

□2 

□2 

□4 

□5 

13/ 

j- 

has  reduced  variations  in  the  way  1  treat 
my  asthma  patients 

□  ; 

□2 

□2 

□4 

□5 

14/ 

SECTION  B:  ABOUT  THE  ASTHMA  TOOLKIT  ITEMS 


3.  Have  you  received  a  copy  or  seen  the  following: 

a.  Pocket  card  containing  the  asthma  guideline 
key  elements 

b.  Documentation  Form  701-R  for  asthma 


(Check  One  Box  on  Each  Line) 


Yes 

No 

□7 

□2 

15/ 

□2 

16/ 

□! 

□2 

17/ 

Master  Problem  Form  702-R  for  asthma 
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d. 

Patient  Action  Plan  Form  703-R  for  asthma 

at 

□2 

18/ 

e. 

Patient  Education  Form  704-R  for  asthma 

at 

□2 

19/ 

f. 

Patient  education  pamphletfs)  for  asthma 

at 

□2 

20/ 

9- 

Patient  education  video  for  asthma 

at 

□2 

21/ 

4.  How  would  you  rate  the  following  toolkit  items? 

(Check  One  Box  on  Each  Line) 


Excellent 

l/e/y 

Good 

Good 

Fair 

Poor 

Not  Used/ Don't 
Know 

a.  Pocket  card 

at 

□2 

□3 

□4 

□s 

□6 

22/ 

h.  Documentation  Form 
701-R 

at 

□2 

□3 

□4 

□s 

□6 

23/ 

c.  Master  Problem  Form 
702-R 

at 

□2 

□3 

□4 

□s 

□6 

24/ 

d.  Patient  Action  Plan 

Form  703-R 

at 

□2 

□3 

□4 

□s 

□6 

25/ 

e.  Patient  Education  Form 
704-R 

at 

□2 

□3 

□4 

□s 

□6 

26/ 

f.  Patient  education 
pamphlet(s) 

at 

□2 

□3 

□4 

□s 

□6 

27/ 

g.  Patient  education 
videos 

at 

□2 

□3 

□4 

□s 

□6 

28/ 

5.  Currently,  do  you  use  the  asthma  outpatient  documentation  form  70  7  -R  to  document  your  care 
of  asthma  patients? 

QYes  CO  TO  Q.  6 
□A/o  GO  TO  Q.  7 

6.  How  often  do  you  fill  out  asthma  outpatient  documentation  form  701-R? 


(Check  One  Box  on  Each  Line) 


Always 

Usually 

Sometimes 

Neyer 

a.  At  first  visit  by  an  asthma  patient 

□  7 

□2 

U3 

□4 

29/ 

b.  At  second  and  subsequent  visits  by 
an  asthma  patient 

□  7 

□2 

□3 

□4 

30/ 

c.  When  asthma  is  a  secondary 
diagnosis 

□  7 

□2 

□3 

□4 

31/ 
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7.  How  often  is  documentation  form  701-R  available  to  you  at  the  following  locations? 

(Check  One  Box  on  Each  Line) 

Always  Usually  Sometimes  Never 


a.  With  patient's  chart  (where  chart  is 

available)  at  time  of  visit  by  an  □/  02  03  04  S2/ 

asthma  patient 


b.  In  the  exam  room 


□  7  a2 


□3 


□4 


SJ/ 


8.  How  often  do  you  document  severity  in  the  patient  medical  chart? 

□  7  At  every  visit 

□  2  Occasionally 

□  3  Never 

9.  Do  you  use  MEDCOM  form  702-R  for  the  patient  action  plan? 

□  Yes  GO  TO  Q.  10 

□  No  CONTINUE  with  Q.  9a 


34/ 


35/ 


9a.  Please  indicate  why  not? 

(Check  one) 

□  7  /  prefer  to  use  my  MTF's  developed  form 

□  2  /  prefer  to  use  my  own  form 

□  3  /  don't  use  a  patient  action  plan 


36/ 


SECTION  C  ABOUT  IMPLEMENTA TION 


70.  Have  you  participated  in  a  formal  training  /  education  session(s)  focused  on  the  asthma 
guideline? 

□  7  Yes-*  CONTINUE  WITH  Q.  10a  37/ 

□2  No-*  GOTO  QUESTION  1 1 

Wa.  Please  rate  how  helpful  the  training  /  education  session(s)  was  for  you. 

(Check  One) 

Extremely  Very  Somewhat  NotHelpful 

Helpful  Helpful  Helpful  at  AH 

□  7  U  2  03  04 


38/ 
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7  7.  Based  on  your  experience,  rate  the  exfenf  to  which  the  following  factors  supported  or 
impeded  your  implementation  of  the  asthma  guideline. 


(Check  One  Box  on  Each  Line) 


Neither  a 

A  major 

A  Minor 

Barrier  nor  a 

A  Minor 

A  Major 

Barrier 

barrier 

Facilitator 

Facilitator 

Facilitator 

a.  Command  support 

□  7 

□2 

□3 

□4 

□5 

39/ 

b.  Credibility  of  guideline 

□  7 

□2 

□3 

□4 

□5 

401 

c.  You  belief  in  value  of 
patient  education 

□  7 

□2 

□3 

□4 

□5 

411 

d.  Competing  priorities  for 
staff  time 

□  7 

□2 

□3 

□4 

□5 

421 

e.  Turnovers  and 

□  7 

□2 

□3 

□4 

□5 

431 

deployments 

f.  Support  of  ancillary  staff 

□  7 

□2 

□3 

□4 

□5 

44/ 

g.  Staff  education  sessions 

1  attended  about  asthma 
guideline 

□  7 

□2 

□3 

□4 

□5 

45/ 

h.  Time  to  process  patients 
under  guideline 

□  7 

□2 

□3 

□4 

□5 

46/ 

i.  Documentation  form 

701 -R 

□  7 

□2 

□3 

□4 

□5 

47/ 

j.  Existing  procedures 

□  7 

□2 

□3 

□4 

□5 

48/ 

k.  Monitoring  of 
performance 

□  7 

□2 

□3 

□4 

□5 

49/ 

12.  Are  you  familiar  with  the  DoD/VA  metrics  (priority  indicators)  recommended  for  monitoring 
implementation  of  the  asthma  guideline? 

□  i  Ves  50/ 


U2  No 
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13.  Have  you  been  given  any  feedback  on  your  MTF's,  clinic's,  or  personal  performance 

regarding  treatment  of  asthma  in  the  last  six  months? 


□  7 

yes-*  CONTINUE  WITH  Q.  13a 

51/ 

□  2 

No-*  GO  TO  Q.  14 

13a.  Please  rate  how  helpful  the  data  was  to  you? 

(Check  One) 

Extremely  Very  Somewhat  Not 

Helpful  Helpful  Helpful  Helpful 

□  7  Q  2  0  3  0  4  S2/ 


SECTION  D:  ABOUT  EFFECTS  OF  THE  ASTHMA  GUIDELINE 


14.  Has  the  asthma  guideline  led  you  to  increase  or  decrease  your  referrals  to  the  following 
services? 


(Check  One  Box  on  Each  Line) 

Not 

Increased 

Greatly 

Increased 

Somewhat/ 

Slightly 

Remained 
About 
the  Same 

Decreased 

Somewhat/ 

Slightly 

Decreased 

Greatly 

Applicable 

or 

Available 

a.  Asthma  education 
centericlasses 

□  I 

□  2 

□  2 

□4 

□  5 

□  6 

53/ 

b.  Spirometry 

□  I 

□  2 

□  2 

□  4 

□  s 

□  6 

54/ 

c.  Test  for  airway 
hyperresponsiveness 
methacholine  challenge 

□  I 

□  2 

□  2 

□4 

□5 

□  6 

55/ 

d.  X  ray 

□  ; 

□  2 

□  2 

□4 

□  s 

□  6 

56/ 

e.  Respiratory  therapy 

□  ; 

□  2 

□  2 

□4 

□  s 

□  6 

57/ 

f.  AllergistI  immunologist 

□  I 

□  2 

□  2 

□  4 

□  s 

□  6 

58/ 

g.  Pulmonologist 

□  ; 

□  2 

□  2 

□  4 

□  s 

□  6 

59/ 

h.  Other  asthma  specialist 

□  ; 

□  2 

□  2 

□4 

□  s 

□  6 

60/ 

i.  Case  manager 

□  ; 

□  2 

□  2 

□4 

□  s 

□  6 

61/ 

j.  MEB  review 

□  / 

02 

02 

04 

Os 

06 

62/ 

k.  Smoking  cessation 
program 

□  7 

02 

□  3 

□  4 

Os 

06 

63/ 
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15. 


Do  you  agree  or  disagree  with  the  following  statements  about  effects  of  the  asthma  guideline 
on  patients? 

(Check  One  Box  on  Each  Line) 


Strongly 

Agree 

Agree 

Neither 
Agree  nor 
Disagree 

Disagree 

Strongly 

Disagree 

Don't 

Know 

a. 

Asthma  patients  are  now 
more  satisfied  with 
the  care  they  receive 

□  / 

□  2 

□  3 

□4 

□5 

□  6 

64/ 

b. 

Asthma  patients  complain  more 
frequently  that  they  do  not  get  the 
treatment  they  expect 

□  ; 

□  2 

□  3 

□  4 

□  s 

□  6 

65/ 

c. 

Asthma  patients  take  more 
responsibility  for  their  care 

□  / 

□  2 

□  3 

□  4 

□  s 

□  6 

66/ 

d. 

Asthma  patients  are  less  likely  to 
require  emergency  treatment 

□  ; 

□  2 

□  3 

□  4 

□  s 

□  6 

67/ 

16.  Does  the  guideline  make  the  following  specific  recommendations  about  the  treatment  of 
asthma  patients? 

(Check  One  Box  on  Each  Line) 


Yes 

No 

Don 't  Recall 

a. 

Recommend  use  of  spirometry  at  least  every  six  months 

□  7 

□2 

□2 

68/ 

b. 

Provide  asthma  patients  with  written  action  plans 

□  7 

□2 

□2 

69/ 

c. 

Classify  asthma  severity  at  every  visit 

□  7 

□2 

□-5 

70/ 

d. 

Prescribe  long  term  control  medication  for  mild  intermittent 
asthma 

□  7 

□2 

□2 

71/ 

1 7.  The  guideline  has  influenced  my  prescribing  of  pharmaceuticals  in  the  management  of 
acute  asthma  patients  in  the  following  ways: 


(Check  One  Box  on  Each  Line) 

increased  Prescribing  No  Change  in  Decreased  Prescribing 


Prescribing 


a. 

Long-term  controller 
medications 

□  7 

□2 

□3 

72/ 

b. 

Oral  corticosteroids 

□  7 

□2 

□3 

73/ 

c. 

Inhaled  steroids 

□  7 

□2 

□3 

74/ 

d. 

Inhaled  short  acting 
beta  2  agonists 

□  7 

□2 

□3 

75/ 

e. 

Long  acting  beta  2 
agonists 

□  7 

□2 

□3 

76/ 

f 

Leukotrene  modifiers 

□  7 

□2 

□3 

77/ 

SECTION  E:  ABOUT  YOU 


18.  Are  you: 

7  □  Maie  78/ 

2  □  Female 

19.  Are  you  a: 

(Check  One) 

7  □  Family  Practice  Practitioner  79/ 

2  □  Internist 

3  □  Other  (specify) _  80-81/ 
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20.  How  long  have  you  been  practicing  medicine? 


Years: 


21.  How  long  have  you  been  stationed  at  this  MTF? 


Months: 


22.  To  which  location  are  you  primarily  assigned? 
(Check  One) 

]  □  Clin  ic 
2  □  TMC 
3U  ER 

4  □  Other  (specify) _ 


23.  The  purpose  of  the  following  questions  is  to  obtain  from  you  an  assessment  of  your  feelings 
regarding  practice  guidelines  and  their  use  in  medicine. 


7 - 

Strongly 

- -2 . -3 . 4 - 

Disagree  Slightly  Neither 

- 5. - 6 - 

Slightly  Agree 

- 7 

Strongly 

Disagree 

Disagree 

Agree 

Agree 

1.  Practice  guidelines  oversimplify  diagnostic  and  treatment  decisions  in  medicine. 

2.  Practice  guidelines  could  help  me  deliver  better  patient  care. 

3.  Use  of  practice  guidelines  In  medicine  will  limit  a  physician 's  freedom  to  take  action. 

4.  Practice  guidelines  help  reduce  variation  In  clinical  practice. 

5.  Use  of  practice  guidelines  will  reduce  provider  efficiency. 

6.  Use  of  practice  guidelines  Is  a  good  way  to  summarize  and  reinforce  scientific 
evidence  on  diagnosis  and  management  of  specific  conditions. 


THANK  YOU  FOR  COMPLETING  THIS  SURVEY. 


82-83/ 

84-85/ 

86/ 

87-88/ 


89/ 

90/ 

91/ 

92/ 

93/ 


94/ 
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Analyses  of  Asthma  Metrics 


To  test  for  effects  of  the  introduction  of  the  DoD/VA  asthma  guide¬ 
line  on  service  utilization  and  prescription  patterns,  we  fit  a  series  of 
regression  models  to  predict  effects  on  each  of  the  six  measures  for 
asthma  treatment.  We  present  in  this  appendix  tables  with  descriptive 
statistics  for  each  measure  (Tables  E.  1  through  E.6).  Results  of  the 
regression  models  that  tested  for  possible  effects  of  guideline  imple¬ 
mentation  on  trends  for  each  measure  are  reported  in  Chapter  Five. 

Table  E.1 

Percentage  of  Asthma  Patients  Prescribed  Long-Term 
Controller  Medications  for  Target  Demonstration,  Other 
Demonstration,  and  Control  Groups,  by  Year 


Group 

Study  Year  One 

Study  Year  Two 

Target  demonstrations 

50.9 

42.8 

Other  demonstrations 

32.5 

29.9 

Control  sites 

43.7 

40.6 

Table  E.2 

Percentage  of  Asthma  Patients  Prescribed  Complementary 
Medications  for  Target  Demonstration,  Other 
Demonstration,  and  Control  Groups,  by  Year 


Group 

Study  Year  One 

Study  Year  Two 

Target  demonstrations 

27.1 

17.3 

Other  demonstrations 

20.3 

14.1 

Control  sites 

19.9 

15.9 
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Table  E.3 

Percentage  of  Asthma  Patients  Prescribed  Short-Acting 
Rescue  Medications  for  Target  Demonstration,  Other 
Demonstration,  and  Control  Groups,  by  Year 


Group 

Study  Year  One 

Study  Year  Two 

Target  demonstrations 

77.0 

72.8 

Other  demonstrations 

59.2 

56.8 

Control  sites 

65.3 

65.0 

Table  E.4 

Average  Annualized  Asthma-Related  Outpatient  Visit  Rates  per  100 
Asthma  Patients  for  Target  Demonstration,  Other  Demonstration,  and 
Control  Groups,  by  Quarter 


Facilities 

Study  Year  One 

Study  Year  Two 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Target  demonstrations 

242 

■a 

WM 

la 

la 

172 

187 

Other  demonstrations 

151 

■a 

mm 

wm 

la 

96 

134 

Control  sites 

199 

168 

139 

158 

193 

la 

144 
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NOTE:  Annualized  outpatient  visit  rates  are  calculated  as  four  times  the  number 
of  visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant  study  year 
and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per  100 
patients. 

Table  E.5 

Average  Annualized  Asthma-Related  Emergency  Room  Visit  Rates  per 
100  Asthma  Patients  for  Target  Demonstration,  Other  Demonstration, 
and  Control  Groups,  by  Quarter 


Facilities 

Study  Year  One 

Study  Year  Two 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Target  demonstrations 

25 

24 

28 

30 

30 

17 

21 

20 

Other  demonstrations 

29 

23 

23 

29 

29 

27 

19 

30 

Control  sites® 

39 

26 

30 

39 

36 

17 

21 

24 

^Excludes  control  site  6,  which  does  not  have  an  emergency  room. 

NOTE:  Annualized  emergency  room  visit  rates  are  calculated  as  four  times  the 
number  of  visits  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant 
study  year,  and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per 
100  patients. 
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Table  E.e 

Average  Annualized  Asthma-Related  Hospitalization  Rates  per  100 
Asthma  Patients  for  Target  Demonstration,  Other  Demonstration,  and 
Control  Groups,  by  Quarter 


Facilities 

Study  Year  One 

Study  Year  Two 

1st 

2nd 

3rd 

4th 

1st 

2nd 

3rd 

4th 

Target  demonstrations 

6 

6 

7 

8 

7 

5 

6 

7 

Other  demonstrations 

6 

6 

6 

10 

5 

3 

4 

4 

Control  sites 

7 

5 

5 

6 

6 

5 

5 

7 

NOTE:  Annualized  hospitalization  rates  are  calculated  as  four  times  the  number  of 
inpatient  stays  in  a  quarter  divided  by  the  number  of  patients  in  the  relevant 
study  year,  and  applying  100  as  an  adjustment  factor  to  standardize  to  a  rate  per 
100  patients. 
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